Discuss this article on the forums
By Parvofighter
PART 1
Much has been made of the “well-characterized” or “well-defined” patient cohorts (https://www.bmj.com/cgi/content/abstract/340/feb25_1/c1018) of the failed XMRV-in-ME/CFS (Myalgic Encephalomyelitis/Chronic Fatigue Syndrome) replication attempts. “Well characterized” somberly implies that the patients studied were robustly and meaningfully comparable to those in the landmark Science XMRV study. Namely: Canadian/Fukuda Criteria Chronic Fatigue Syndrome patients with severe disability, reproducible immune abnormalities, cognitive deficits, prolonged, disabling fatigue, low V02 max, and multiple longitudinal measurements of clinical and laboratory abnormalities (https://www.sciencemag.org/cgi/content/full/1179052/DC1 ).
“Well-characterized” carries an aura of respectability – defying readers to challenge the issue of cohort selection in XMRV research. And it has been abundantly used in the checkered history of Chronic Fatigue Syndrome research. The latest well-characterized XMRV study in Chronic Fatigue Syndrome was ushered into the expectant scientific community by the esteemed British Medical Journal. In her Editor’s Choice letter (https://www.bmj.com/cgi/content/full/340/mar04_1/c1266?maxtoshow=&hits=10&RESULTFORMAT=&fulltext=xmrv&searchid=1&FIRSTINDEX=0&sortspec=date&resourcetype=HWCIT ) accompanying the latest XMRV retroviral study by psychiatrists (Van Kuppeveld et al), Fiona Godlee empathetically waxed poetic on the roller coaster of emotions ME/CFS patients had endured since the BMJ’s publication of the latest rebuttal of XMRV/CFS research:
“Meanwhile, and sadly for those whose hopes had been raised, the (Science) study has been refuted by three further case-control studies, one of them in the BMJ (doi:10.1136/bmj.c1018).”
With a funereal tone, Godlee cited the somber news that, “claims of association between retroviruses and diseases often fail to withstand the test of time (doi:10.1136/bmj.c1099).” AIDS patients around the world bowed their heads in silence as the coffin began to close on XMRV’s association with ME/CFS.
Godlee went on to provide a thrilling and compelling rationale for the BMJ’s publication of the latest XMRV research:
“The paper by Van Kuppeveld and colleagues is an unusual paper for the BMJ to publish. As our research highlights page explains, we would usually reject a small case-control study examining the prevalence of a virus in 20 year old blood samples. Instead we fast tracked it. We did this because it’s about an important and debilitating syndrome that’s often seen by generalists and because we felt it added to an important and highly controversial debate. We and our reviewers
also thought it was well done.”
Godlee gamely heralded a new era of scientific excellence for Chronic Fatigue Syndrome and XMRV, entitling her missive with the serious invocation: “Let’s Proceed with Caution”. Godlee wisely enlisted the help of experts: Cathie Sudlow stepped up to the plate to help her navigate the minefields of credible scientific enquiry:
“As an epidemiologist, Cathie Sudlow’s initial response was skeptical, quickly confirmed when she saw that the paper lacked basic methodological information. “Where were the details of the characteristics and selection procedures for the cases and controls”, or of blinding of researchers to the case-control status of the samples? Where was the discussion of the potential role
of bias and confounding?”.
Godlee may be handicapped when it comes to irony, and Sudlow, when it comes to reading Science’s Supporting Online Materials (www.sciencemag.org/cgi/content/full/1179052/DC1 ) but they are a force to be reckoned with when it comes to promoting quality scientific research. Godley closed her tome on XMRV and ME/CFS with a flourish:
So yes, let’s have more research into chronic fatigue syndrome,
but let’s make sure it’s good enough research.
Despite her evident talents as a BMJ editor, Godlee is subtly handicapped when it comes to her sense of irony. But this doesn’t deter her zealous scientific pursuits. Godlee’s, and indeed the BMJ’s enthusiasm for quality medical research is iron-clad – particularly when it is associated with a retrovirus believed to cause devastating neuro-immune disease and cancer – and this we enthusiastically applaud.
In Part 2, let’s next explore the wonders of Van Kuppeveld et al’s BMJ XMRV work, and the source of their highly credible patient cohort.
BMJ and XMRV Unplugged:
PART 2
A “well-defined” patient cohort
Where did Van Kuppeveld’s patients come from?
Enthralled by the British Medical Journal’s unbiased coverage of XMRV and Chronic Fatigue Syndrome, I began a quest to learn more about these hapless patients. Where did they come from? What did they do to earn the distinction of being a “well-defined patient cohort”, as described in the BMJ article’s Study Design section (https://www.bmj.com/cgi/content/abstract/340/feb25_1/c1018)? After all, the Chronic Fatigue Jihadists – those nasty ME/CFS patients were nipping annoyingly at my heels.
My first stop was the Methods section of Van Kuppeveld et al’s BMJ masterpiece: Prevalence of xenotropic murine leukaemia virus-related virus in patients with chronic fatigue syndrome in the Netherlands: retrospective analysis of samples from an established cohort. https://www.bmj.com/cgi/content/full/340/feb25_1/c1018
Methods… All patients and controls examined in this study were part of a Dutch cohort of 298 patients, which has been described in detail (2 citations provided below). All patients of this cohort fulfilled the Oxford criteria….
With grateful credit to the genuinely brilliant Gerwyn on the Phoenix Rising Forums (,https://www.forums.aboutmecfs.org/showthread.php?3554-The-origin-of-the-patients-in-the-Dutch-XMRV-study.&highlight=vercoulen ), I then turned my attention to the two papers cited as sources for the patients for the BMJ Van Kuppeveld study. These were:
a) Vercoulen JH, Swanink CM, Fennis JF, Galama JM, van der Meer JW, BleijenbergG. Dimensional assessment of chronic fatigue syndrome. J Psychosom Res 1994;38:383-92. https://dare.ubn.kun.nl/bitstream/2066/14900/1/4783.pdf
b) Swanink CM, Vercoulen JH, Galama JM, Roos MT, Meyaard L, van der Ven-Jongekrijg J, et al. Lymphocyte subsets, apoptosis, and cytokines in patients with chronic fatigue syndrome. J Infect Dis 1996;173:460-3. https://pubmed.ncbi.nlm.nih.gov/8568312
BMJ XMRV patients sourced from a study in the Journal of Psychosomatic Research!
I like to know who I’m reading about, so I was thrilled to learn on PubMed that J.H. Vercoulen, lead author in the prime study where patients were obtained, had practiced over many years at the Department of Medical Psychology at University Hospital Nijmegen/ Radboud University Nijmegen Medical Center in The Netherlands. I was also pleased to see that of the BMJ XMRV/CFS authors, at least three of them (Swanink, Galama , and van der Meer) had participated in the source studies. What luck!
This guaranteed that the BMJ XMRV research team would be intimately familiar with the patient cohort characteristics they drew
from for their CFS study!
Dispensing with the Swanink Study
I was able to dispense quickly with the Swanink study (https://pubmed.ncbi.nlm.nih.gov/8568312) because all they did in terms of their cohort, was to randomly select them from the Vercoulen study:
Seventy-six patients and 69 healthy controls matched for sex, age, and neighborhood were included in the study. As described before, patients were randomly chosen from a data base of 298 patients with CFS using a table of random numbers.
https://www.jstor.org/pss/30126146
The database of 298 patients used for the XMRV/CFS research: The Vercoulen, Journal of Psychosomatic Research Study!
Now for the source of these well characterized patients! Given how suspicious the BMJ is about research quality, I thought it wise to include actual screen captures of this cohort description. All excerpts are from: the Vercoulen et. al. article from the Journal of Psychosomatic Research: https://dare.ubn.kun.nl/bitstream/2066/14900/1/4783.pdf Here we go…
Of note was the last sentence: “These inclusion and exclusion procedures resulted in a study sample of 298 subjects.”
395 self-referred patients
Patients with definite or suspected illnesses causing fatigue excluded
Patients taking heart meds excluded – i.e. many advanced ME/CFS patients excluded
Final study sample of 298
Gerwyn on the PR Forums was audacious enough to question the self-referral and telephone diagnosis process. Surely this was on par with the Science patient cohort procedures!
Addressing the BMJ and Cathie Sudlow’s concerns about bias and confounding
I was gratified that the BMJ XMRV article’s patient samples came from a study from the esteemed Journal of Psychosomatic Research. As a peer-reviewed journal, this would ensure that any bias or confounding factors – a valid concern explicitly raised by the BMJ – would be summarily dealt with. And after all, the British Medical establishment has an illustrious history of cooperating with psychiatrists in the pursuit of a biomedical cause for ME/CFS. Just ask Simon Wesseley. Fortunately, it was clear that CFS patients seen in the Vercoulen source cohort were not treated with any bias. A wonderful example of this is in the discussion on Avoidance behavior – a well-known tactic of indolent, exercise-averse ME/CFS’ers. Despite years of being jocks, it is indeed perplexing that many ME/CFS’ers characteristically avoid activity like the plague. Almost like a switch went off in their motivation and personality
I also enjoyed the Cognitions and Attributions section, where CFS’ers were asked to mull over the causes of their complaining nature. After all, it is this kind of Cognitive Behavior Therapy-infused atmosphere that causes ME/CFS patients to flock (and self-refer!) to these experts in droves! There’s nothing in a neuro-immune disease, potentially caused by a cancer-causing retrovirus, that a little attitude adjustment can’t help.
Self-reported Symptoms – a hallmark of good XMRV cohort design
There was a slight hiccup in my enthusiasm for the BMJ’s scientific prowess, when I bumbled over Table 1: “Spontaneously reported complaints by 298 CFS-patients”. Here was a list of the various ailments of these patients:
(Picture did not come through: suffice it to say they did not look like typical CFS patients).
It was a little disturbing that a group that otherwise so perfectly matched the Science cohort with reproducible immune abnormalities, would only have 26% of patients with recurrent infections. And maybe moreso that patients didn’t mention or even describe post-exertional malaise, the pathognomic sign for Canadian-Criteria ME/CFS. Then again, the Vercoulen et al work predated the Canadian Criteria. Ah, but that’s just one of the pitfalls when you’re living in the fast lane with BMJ. Be reasonable – world-class scientists researching the next AIDS can’t be expected to wait to use the Canadian-Criteria when they’re in a rush – and have 16-year old Oxford Criteria blood handy!
PART 3
Power to the people: Self-reported physical findings
I then reminded myself that all these symptoms were self-reported, and my concerns about scientific integrity at the BMJ were quickly assuaged.
INSERT VERCOULEN 94-09
It was wonderful to see the patients given so much latitude to define their own symptoms. Far simpler too, for them to do this as a take-home questionnaire, and to self-report their medical status, than to submit to all those pesky “longitudinal measurements of clinical and laboratory abnormalities” that those Yanks used. (https://www.sciencemag.org/cgi/content/full/1179052/DC1 ).
Psychological well-being
Things were going really well. Remember the voice of the BMJ Swat Team of reviewers when they evaluated the Van Kuppeveld paper?
“We and our reviewers also thought it was well done.”
https://www.bmj.com/cgi/content/full/340/mar04_1/c1266?maxtoshow=&hits=10&RESULTFORMAT=&fulltext=xmrv&searchid=1&FIRSTINDEX=0&sortspec=date&resourcetype=HWCIT
Long live the British Empire! I just knew that we could count on a scientifically robust evaluation on XMRV Science from unbiased reviewers like the BMJ … and the global media! So it could be considered just a slight indiscretion that the Van Kuppeveld team slipped in a patient cohort where 36% met the criteria for clinical depression. Yes, you heard that right! But don’t worry…
“Using a score of 16 or more, 36% of patients could be
considered as having a clinical depression.”
Now first of all, I wanted to check the numbers. The Vercoulen et al paper had 298 patients (after all exclusions were performed); the 2nd source paper by Swanink et al had 76 CFS patients. BUT these Swanink patients were drawn from Vercoulen’s original 298. And since these were randomly drawn, the same % (36% clinically depressed patients) can be expected to hold.
I actually checked the front cover of the Vercoulen et al paper to make sure that I wasn’t seeing things. But then I realized that I was probably just being paranoid. Or depressed. What was the big deal if 36% of the BMJ XMRV/CFS cohort had clinical depression? There was no reason for me to doubt the medical establishment – or even the Chronic Fatigue Immune Dysfunction Association of America – who had encouraged patients and the international media to relax about pesky cohort issues. I found myself lulled by the prestige with which the BMJ and its coterie are revered. No cause for alarm. All those stern admonishments to the ME/CFS patient community to stop whining about XMRV methodology and cohort selection were well-founded. This BMJ-feedback-loop really works!
Patients with Unexplained Fatigue = Patients with ME/CFS
My education was progressing, and my pulse quickened when I saw in the Discussion that the Vercoulen team were capable of some high-octane self-criticism: the hallmark of any excellent scientist:
“The sample was self-referred.” Well, we already knew that. But what I now learned was that this XMRV cohort was generalizable to other patients with “Unexplained Fatigue”! This was brilliant! After all, every one knows that Unexplained Fatigue is the same as ME/CFS! Even better, there was no qualitative difference in any measure they used (!) between patients with Unexplained Fatigue and the patients (blood) enrolled in the XMRV study… except that the XMRV patients were just a tad worse than your run-of-the-mill Unexplained Fatigue patient. We were in good company! This was panning out to be a perfect match with the Science cohort!
Minimalizing the risk of including patients with delayed convalescence of a viral infection!
It’s a good thing that the BMJ has such a crackerjack team of editors and research consultants! Otherwise I might worry that they missed the part where the Vercoulen team “Minimalized the risk of including patients with delayed convalescence of a viral infection”.
Of course, a sharp epidemiologist like the BMJ’s Cathie would sound off about here:
“Where was the discussion of the potential role of bias and confounding?”.
https://www.bmj.com/cgi/content/full/340/mar04_1/c1266?maxtoshow=&hits=10&RESULTFORMAT=&fulltext=xmrv&searchid=1&FIRSTINDEX=0&sortspec=date&resourcetype=HWCIT
It’s nice that we don’t have to hold the BMJ XMRV researchers to the same standards as those US Science louts. I mean, this is the BMJ we’re talking about. There’s no way that in their desire some 16 years ago to keep out patients with “delayed convalescence of a viral infection”, the CFS researchers of that age might have – unwittingly of course – filtered out by other means too, the patients with any hint of viral etiology. And there’s no way that in their desire to preserve mutually beneficial relationships with renowned psychiatrists and insurance companies, the BMJ might forget to shine their own shoes. I’m glad that we’ve got Cathie Sudlow and Fiona Godlee on our team! This is kinda like flying Singapore Airlines. You can just relax, lay back, and let the BMJ experts take care of investigator bias – and the 3rd human retrovirus!
Fatigue severity – the principal complaint
Everybody knows that ME/CFS is just a lot of plain old tiredness, where patients just need a good kick in the butt! So it wasn’t surpising that when the hotshot statisticians looked at the Vercoulen et al study, they said the following:
In other words, the sadder these BMJ patients were (16 years ago), the less they did, and the less they expected they could control their life: … the more tired they got. That’s what Vercoulen’s data proved. That their patients were depressed. Or at least pretty darn lackluster. This sounds like a patient group in need of a motivational speaker! Or CBT. For all you non-statisticians (including me), R2 is just a fancy way of saying, “is there a linear relationship?” For example, as psychological wellbeing goes down, does the data show us that fatigue severity goes down too? If you have an R2 of 1.0, the answer is YES, WAY! If it’s zero – well, no. The only downside was that the R2 for this argument was kinda wishy-washy at 0.51. And even then, that only explained 27% of the variance.
Actually, at the end of the day, they found that their research wasn’t quite as strong as they hoped:
PART 4
“The combined effect of related dimensions explained only a minor to moderate part of variance”.
In other words, the team was pretty much scratching their heads, although a bit of a trend was becoming evident. Fortunately, if you color-code the various relationships, you can get an idea of what kind of patients these were. First of all, let’s take a look at the table that Gerwyn referenced, Table VI from Vercoulen et al.
(Missing)
Wikipedia (which has its own dramas on the XMRV/CFS research) gave a nice, quick definition of Dependent and Independent Variable:
In a statistics experiment, the dependent variable is the event studied and expected to change whenever the independent variable is altered.
Example: If one were to measure the influence of different quantities of fertilizer on plant growth, the independent variable would be the amount of fertilizer used (the changing factor of the experiment). The dependent variables would be the growth in height and/or mass of the plant
https://en.wikipedia.org/wiki/Dependent_and_independent_variables
So in the chart above, they’re saying that the key symptom, Fatigue Severity, is influenced (somewhat) by Psychological wellbeing, Functional Impairment, and Self-Efficacy.
Is there a way to make all of this make sense, so we can see if the 16-year old blood in the BMJ Van Kuppeveld paper had the same (or at least a VERY similar) cohort to the Science patients? Out of interest, I drew a flow-chart, starting with Fatigue Severity, following the items in Table VI, and working from Fatigue Severity toward the right hand side of the page. I color-coded items that were pretty much psychological in yellow. Items that were mixed physical/psychological I put in yellow/green rainbow. This whole research paper was all subjective, so I was in good company. I was going to put pure physical items in green. But unfortunately I couldn’t find any, as in the Science cohort (such as 2-5A Synthetase/RNase-L deficiency or V02 Max abnormalities, or low Natural Killer cell cytotoxicity measures.) So I kept things simple.
First things first, here’s a guide to the abbreviations:
Yellow = mainly Psychological items
ψWb= Psychological wellbeing (using the Greek letter psi or ψ)
SEf = Self-efficacy
SoF = Social functioning
CAt = Causal attributions (what do you blame your tiredness on)
APA = Avoidance of physical activity
Yellow/green = mixed Psychological/Physical
FS = Fatigue severity (the main symptom according to them)
FI = Functional impairment (things you can’t do)
Con = Concentration
SDi = Sleep disturbances
No Green (Pure Physical) items
Like 2-5A Synthetase/RNase-L deficiency
Or V02 Max abnormalities
Or low Natural Killer cell cytotoxicity measures
I ignored the perfection of the regression line (R2 = 0 to 1) and just focused on the percentage of variance explained in these relationships. After all, this was just for illustrative purposes. The bottom line is that if you follow these unspectacular linkages all the way to the right side of the page, a very rough estimate is that 70% of the factors that can be identified to drive Fatigue Severity in the BMJ’s cohort are psychological, and the remaining 30% are partly psychological. Which rather nicely summarizes the perspectives of many CFS researchers 16 years ago – and even today. And reinforces the British and indeed Dutch paradigm to deny ME/CFS patients physical testing and diagnostics. Why bother when it’s all psychosomatic?!
Bottom line, there are no pure green boxes which might capture potential biological drivers of ME/CFS. Or which might give a hint (think RNase-L deficiency) that the cohort is somehow vulnerable to viral infections. The green box is devoid of the kinds of physical findings that the Science cohort was characterized by, and which the term “Well Characterized” and “Well defined” imply.
At this point, I conceded that I was indeed depressed. And sorely disappointed. And “like totally surprised” that my friends at the BMJ came up lacking. And I realized that the BMJ’s scientific integrity on the XMRV/ME/CFS research wasn’t just unplugged. It was defrocked! …
What do you think?
PART 5 of 5 : The BMJ Defrocked
Is it meaningful that the BMJ cohort came from 16-year old blood and a psychological paradigm, infused with self-referred CBT enthusiasts, self-completed questionnaires, no clinically measured physical signs, and (at least) 36% clinically depressed patients?
Does investigating CFS patients for avoidance behavior, cognitions, and attributions – for a publication in the Journal of Psychosomatic Research, in research led by a psychologist, presuppose a psychosomatic illness?
How fervent were these researchers in their stated aim to “Minimalize the risk of including patients with delayed convalescence of a viral infection”?
Given this explicit aim of the researchers, what evidence is there that investigator bias was eliminated?
Just how zealous were these researchers in excluding patients with signs/symptoms of viral infection?
Would a sane patient with reproducible immune abnormalities and severe, disabling fatigue willingly volunteer for a Cognitive Behavior Therapy program or study to eliminate their “Abnormal Sickness Beliefs”?
Would a sane patient with classic Canadian-Criteria ME/CFS, post-exertional malaise, and severe disabling fatigue even consider a Graded Exercise Therapy program?
Is Unexplained Fatigue the same as ME/CFS?
Has investigator bias been addressed in the Vercoulen paper – the source of the archaic blood for the BMJ study? Has bias been addressed in ANY of the BMJ missives on ME/CFS and XMRV?
Are the Science and BMJ XMRV cohorts even faintly comparable?
Is the BMJ’s work on XMRV and ME/CFS worth the paper it’s written on… Or worth the energy to power the pixels on your screen?
You tell me.
Wind back to Dr Vernon of the Chronic Fatigue Immune Dysfunction Association of America on the BMJ study:
“XMRV was not detected in a third follow-up study (BMJ) from a well-characterized cohort of CFS subjects.”
https://www.cfids.org/xmrv/022510study.asp
The PLoS ONE paper by Erlwein, et al, the Retrovirology paper by Groom, et al, and now the van Kuppeveld, et al, paper in BMJ all studied well-characterized patient cohorts that met accepted and widely used CFS case definition criteria.
BMJ Editor Fiona Godlee was right about one thing:
So yes, let’s have more research into chronic fatigue syndrome,
but let’s make sure it’s good enough research.
YES- feel free to post this elsewhere. Kindly give
credit to the source (just input the link below)
https://www.forums.aboutmecfs.org/showthread.php?t=3860