• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Validating a measure of ME/CFS symptomatology - Brown & Jason (DePaul Symptom Questionnaire)

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Validating a measure of myalgic encephalomyelitis/chronic fatigue syndrome symptomatology
Abigail A. Brown & Leonard A. Jason
Fatigue: Biomedicine, Health & Behavior: Vol. 2, Iss. 3, 2014
Published online: 23 Jul 2014
http://www.tandfonline.com/doi/full/10.1080/21641846.2014.928014#.U9E8mvldWSo

This paper works towards validating the DePaul Symptom Questionnaire.
Unfortunately the full paper isn't available for free.

Abstract
Background: The diagnosis of myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) is complex and largely based on self-reported symptom profiles. The field lacks consensus for a singular case definition and heterogeneous samples make comparability across studies difficult.
Purpose: The present study sought to validate a comprehensive self-report measure of ME/CFS symptomatology to aid in clinical and research assessment.
Methods: Exploratory factor analysis (EFA) was used to establish the underlying factor structure of the DePaul Symptom Questionnaire (DSQ) using a well-characterized sample of individuals (92.6% met the Fukuda et al. criteria and/or the Clinical Canadian Criteria) and this structure was then tested on a less stringently recruited sample of individuals utilizing a confirmatory factor analysis (CFA). Convergent and discriminant validity of the DSQ were also examined utilizing alternative measures of symptomatology and functioning.
Results: A three-factor solution was found using EFA (Neuroendocrine, Autonomic, and Immune Symptoms; Neurological/Cognitive Dysfunction; Post-Exertional Malaise) and the fit of this factor structure was adequate for the second sample. The DSQ was found to have good convergent and discriminant validity.
Conclusions: The DSQ is a valid tool for assessing ME/CFS symptoms. There may be two core ME/CFS symptom clusters: post-exertional malaise and cognitive dysfunction.
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Hope to get to see the full text of this
Results: A three-factor solution was found using EFA (Neuroendocrine, Autonomic, and Immune Symptoms; Neurological/Cognitive Dysfunction; Post-Exertional Malaise) and the fit of this factor structure was adequate for the second sample
The crucial thing about all these fancy factor analyses is that they hold up in independent samples, as opposed to being driven by the data in one specific sample. And the 'adequate' description of performance in a second sample isn't exactly reassuring. Though hard to judge from an abstract alone.

However, the 3-factors identified were previewed in Lenny Jason's presentation to IOM, 5 May 2014 (powerpoint presentation)

Brown and Jason’s (2014 - ie this published paper, then a submitted manuscript) Study Identified Three Factors:

jason 3 factor.png


Interpretation of Brown et al. Study
•Two of the emergent factors were
–Neurological/Cognitive Dysfunction
–Post-Exertional Malaise
–fit well with previous literature indicating that these are two of the cardinal symptom clusters of ME/CFS

•One factor was items from Neuroendocrine, Autonomic, & Immune Dysfunction
more difficult to interpret as it incorporates many symptom clusters

•This suggests that there may be core, well-defined symptom clusters such as cognitive impairment and post-exertional malaise
–but also that there may be many other symptoms that are experienced differently by patients
=====

All of which looks pretty interesting to me: PEM + Neurocognitive as core symptoms common to all patients, plus a group of other symptoms experienced in different selections by different patients.

added: fun video! Factor analysis in under 2 mins. Basically it's about simplifying the data by finding a small number of underlying common 'factors' that causes individual items to cluster together. Guaranteed maths-free

online slideshow with quite a lot of detail: Exploratory factor analysis
nice quote from it
The common clusters (called factors) are summary indicators of underlying 'fuzzy constructs'
The whole process of Factor Analysis is very fuzzy, with a lot of scope for, even requirement for, researcher judgement during the analysis (choosing different forks in the path). That's why replication of any factor structure on an independent sample is important.
 
Last edited:

Dolphin

Senior Member
Messages
17,567
One general point about factor analysis is the names that are given to factors can be somewhat arbitrary. I've come across cases where a researcher gave particular factors names which made it sound all nice and exact but when one looked at the individual entries, they didn't always match the names for the groups.
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Commentary on full text, focusing on symptom clusters
Validating a measure of ME/CFS symptomology
[see top of page for abstract]​

Thanks to Prof Lenny Jason for the full text and for answering questions.

Although this study makes lots of interesting points about case definitions, its primary purpose was to validate the DePaul Symptom Questionnaire (DSQ). The questionnaire was broadly validated (I'll spare you the tedious statistical details), though the team are planning on tweaking it in the light of these results.

The intro criticises Fukuda for its polythetic criteria ie no specific mandatory symptoms beyond unexplained fatigue (basically: 'Pick four symptoms. Any four will do'). Then focuses on case definition work by Ramsay, Goudsmit, dowsett and Hyde:
Taken together, these theorists propose a more narrow view of ME and the emergent criteria require an individual to experience post-exertional malaise, at least one neurological symptom, at least one autonomic symptom, and the onset of the condition had to have been sudden (developing over one week or less).
Jason prefers this approach to the more extensive symptoms required by CCC and ICC. But ultimately believes we need data-driven approaches to find the most appropriate real-world definition.

The samples: Biobank and DePaul
The main sample used was of 214 subjects from the CFIDS biobank.These were all "diagnosed by a licensed physician specializing in CFS, ME/CFS and ME care". 74% were female and average age 49. Only 12% were working full or part-time, with 67% on Disability benefit. However, 67% had completed college and 24% also had a graduate or professional degree so this sample is not representative educationally. Note that people volunteer for this, rather than being a random selection and that may skew the results (this educational skew to graduates is true of many studies too eg the recent Andrew Miller fMRI one).

The DePaul sample (n=189) used to validate the results was a convenience sample, relied on self-report of current ME/CFS rather than physician diagnosis and 40% had a graduate degree (76% any degree), so is not exactly ideal.

Results: 3 clusters, modest fit with the data
As covered before, they found three factors/clusters:
  • Neuroendocrine, Autonomic, & Immune Dysfunction
  • Neurological/Cognitive Dysfunction
  • Post-Exertional Malaise
index.php


These tie in broadly with the three symptoms highlighted by the theorists: PEM, neurological and autonomic.

However, the 'fit' isn't that good. The success of Factor Analysis in identifying strong clusters is measured by "percentage of variance explained" - or how well the clusters account for the actual data. 100% is a perfect result (so never seen) and 0% is the worst. Typically, 65% or more is seen as very good, and 50% is reasonable.

However, in this study the three clusters only accounted for 42% of the data. The biggest was the broad broad autonomic neuroendocrine/immune cluster accounting for 31% for the - with only 5% accounted for by PEM.

In stage two of the study, they looked to see how well these three factors derived from the BioBank data fitted the DePaul cohort data. While I have a reasonable understanding of the 'explanatory factor analysis' used in stage one, the 'confirmatory factor analysis' used in stage two is beyond me - so I will take the authors word for it that the fit was "adequate" i.e. OK, but not exactly impressive.

In summary, it looks like the authors are on to something important about how symptoms cluster,
but they haven't exactly nailed it yet.

What this means/Next steps
A weakness that the authors mention is that the sample size is on the small size for this particular technique. There is debate about how big a sample is needed, but given the number of symptoms included in this analysis, a sample size of roughly 400 or more would probably be ideal (vs 214 used in stage one).

The good news is that the team are already working on a new analysis using larger datasets.

Another reason the clusters aren't that strong is that symptoms might not be enough to classify people - maybe you need objective tests too in order to reveal what's really going on.
The authors said:
Future studies might be directed towards validating this DSQ using more objective measures. For example, performance on neuropsychological measures could be related to neurocognitive factor-based scores, and immunologic testing following exercise testing could be associated with post-exertional malaise factor-based scores

Finally, the authors emphasise their view that case definitions should be driven by data:
The current findings may contribute to the development of a data-driven case definition for this illness. Future work in this field should continue to utilize symptom data from well-characterized patient samples as the basis of case definition, rather than relying on clinical consensus.
To which I would simply add their earlier point about using objective markers alongside symptoms.
 
Last edited: