• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of PACE trial

Bob

Senior Member
Messages
16,455
Location
England (south coast)
They seem to have used a very complex description of the techniques they used which suggests to me they may not get them and have not really thought out the problem in detail. Stats help us think about data it never provides an algorithm where you plug in data and get results. Thought about what is going on and hence what the algorithms say is necessary.
Based on their past record, I would place money on none of the authors having a proper understanding of the statistical analysis (or even a superficial understanding of it) except for the professional statistician.
 
Last edited:

Bob

Senior Member
Messages
16,455
Location
England (south coast)
If anyone could correct me if Im wrong but superficially it seems like:

The 6MWT whilst showing statistically significant improvement - the real world improvement for improving the illness is practically negligible.
I think that is a good characterisation. We think that the overall improvements for GET vs SMC were not clinically useful, and had an effect size in the 'small' range which is generally not considered to be clinically useful. The new paper says that changes in the 6MWT for GET were 'significant', but that seems to mean just that it was a statistically significant change. For CBT vs SMC there was no improvement in the 6MWT.

The rest of the evidence for GET and CBT is take from simplistic subjective assessments - and again - is very slight.
Their main primary outcome measures were both self-report outcomes. The SF-36 physical function scale is a widely used self-report scale. These aren't necessarily meaningless outcomes, but they have to be interpreted in context. The primary outcomes had a moderate effect size, which is considered useful, but disappointing. But, again, this outcome has to be considered in context. Considering that the trial was open-label, and without a placebo control, it's possible that all of the improvements could be explained by a placebo effect (i.e. that you would expect in a homeopathy study), or explained by biases inherent in the study methodology.

Is there a log anywhere for the claims and press released around the PACE announcements? I am interested in making a comparison chart with what they said, what the press printed - and what the real results were...
This is a bit complex, because they have said lots of things in relation to PACE. I'm not sure what you are looking for. I don't know if there is a brief summary of the PACE Trial outcomes anywhere.

I do have a list of newspaper reports relating to the initial 2011 PACE Trial report:

---------------
The Times (paywall)
ME sufferers ‘better pushing their limits’
18 February 2011
http://www.thetimes.co.uk/tto/health/news/article2917876.ece

"Talking therapy and exercise both proved more effective than treatments which “pace” the patient’s activity to their energy levels, a study in The Lancet concludes, with twice as many patients making a full recovery."
"About 30 per cent of patients given cognitive behavioural therapy (CBT) or graded exercise made a full recovery to normal levels of activity, the study found..."
---------------
Daily Mail
Got ME? Fatigued patients who go out and exercise have best hope of recovery, finds study
18 February 2011
http://www.dailymail.co.uk/health/a...-exercise-best-hope-recovery-finds-study.html

"The results showed that CBT and GET benefited up to 60 per cent of patients, and around 30 per cent of patients in each of these treatment groups said their energy levels and ability to function and returned to near normal levels."
The Daily Mail reported that ME patients should “push themselves to their limits” for the “best hope of recovery”.
---------------
The Guardian
Study finds therapy and exercise best for ME
18 February 2011
http://www.guardian.co.uk/society/2011/feb/18/study-exercise-therapy-me-treatment

"The biggest-ever study of treatments for ME, also known as chronic fatigue syndrome, has found that more people recover if they are helped to try to do more than they think they can – rather than adapting to a life of limited activity."
---------------
The Independent
Got ME? Just get out and exercise, say scientists
18 February 2011
http://www.independent.co.uk/life-s...-out-and-exercise-say-scientists-2218377.html

"Overall, 60 per cent of patients who received CBT or GET made progress and 30 per cent recovered sufficiently to resume normal lives. Among those who received APT, half as many (15 per cent) resumed normal lives. Fewer than one in ten patients left untreated recover, the researchers said."
---------------
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
This might have been posted before, but I'm still catching up, and it's worth posting again anyway...

From the discussion section of the new paper:
Chalder et al. said:
In an ideal world, mediators would have been assessed at every session. Given the pattern of change in the mediators was similar to the pattern of change in the outcomes it is possible that the variables were affecting each other reciprocally and more measures might have helped clarify this. However, this was prohibitive in this trial because of cost and possible measurement fatigue on the patients' part. Results from longitudinal analyses incorporating all available mediator and outcome measurements will be discussed in future publications. Additionally, we were only able to assess walking at 24 weeks when most of the change in outcomes had occurred. It would have been more convincing as a mediator if walking had been assessed at 12 weeks mid treatment.

So does this confirm that they did not analysis a cause-and-effect relationship between mediators and outcomes? i.e. that mediators and outcomes could have changed simultaneously?
 
Last edited:

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
So does this confirm that they did not analysis a cause-and-effect relationship between mediators and outcomes? i.e. that mediators and outcomes could have changed simultaneously?
They said that mediators and outcomes did change simultaneously, draw your own conclusions about what that means for demonstrating mediation. Also compare fig 3 of the paper ('mediators' over time) with Fig A of the appendix (outcomes).
 
Last edited:

user9876

Senior Member
Messages
4,556
This might have been posted before, but I'm still catching up, and it's worth posting again anyway...

From the discussion section of the new paper:


So does this confirm that they did not analysis a cause-and-effect relationship between mediators and outcomes? i.e. that mediators and outcomes could have changed simultaneously?
I'm surprised (not really) that the reviewers didn't trash the paper given that admission.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
I'm surprised (not really) that the reviewers didn't trash the paper given that admission.
Yes, that admission trashes the rest of the paper, doesn't it! They carried out a complex and apparently sophisticated statistical analysis but that quote makes it all totally redundant!
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
They said that mediators and outcomes did change simultaneously, draw your own conclusions about what that means for demonstrating about mediation. Also compare fig 3 of the paper ('mediators' over time) with Fig A of the appendix (outcomes).
Thanks Simon.

...draw your own conclusions about what that means for demonstrating about mediation.
I have! :)
 
Messages
73
They said that mediators and outcomes did change simultaneously, draw your own conclusions about what that means for demonstrating about mediation. Also compare fig 3 of the paper ('mediators' over time) with Fig A of the appendix (outcomes).

sorry if this sounds stupid but can anyone explain the significance of this? If its important Id like to understand it...

"Overall, 60 per cent of patients who received CBT or GET made progress and 30 per cent recovered sufficiently to resume normal lives. Among those who received APT, half as many (15 per cent) resumed normal lives. Fewer than one in ten patients left untreated recover, the researchers said."
---------------

wow this Independent article is amazing! Are they talking about the same trial?! How could they have made these claims?! Oh I remember - they changed the score system and decided 60 out of 100 means recovery to normal lives...
 
Last edited:

Bob

Senior Member
Messages
16,455
Location
England (south coast)
...can anyone explain the significance of this? If its important Id like to understand it...
I don't know if I will explain it well, seeing as I'm only just getting to grips with it, but here goes...
If the changes in the mediating outcomes (e.g. fear) happened simultaneously to the main trial outcomes (self-report fatigue and physical function), then the investigators haven't demonstrated that one led to the other, but they are just making an assumption that a reduction in fear has led to an improvement in the primary outcomes. In fact, if the changes happened simultaneously, then it could have been the other way around: an improvement in fatigue and physical function could have mediated a reduction in fear (which makes more sense to me.) Or there might not be any direct relationship. In relation to all of this, they say: "it is possible that the variables were affecting each other reciprocally", which is code for "we really haven't got a clue, and we just made it up, but hopefully no one important will notice".

It's all a load of nonsense, of course, seeing as all they've proven is that CBT and GET are equivalent to homoeopathy or a placebo, with no improvements seen in any objective measures.
 
Last edited:

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Some notes.

Analysis - individual or average outcomes?
Despite having read the whole paper, but not understanding all of it, I still don't know if the analysis is based on individual or average scores.

CBTvsAPT and GETvsAPT - valid comparison?
APT, was a previously untested novel therapy designed especially for the PACE trial, which caused a reduction in physical function (but not fatigue) when compared with SMC-alone. i.e. it had a negative effect for physical function when compared with the SMC control group.
So, considering that APT impaired improvements, a comparison with CBT or GET will place CBT/GET in a favourable light.
As an extreme example, it's a bit like giving patients arsenic, and then declaring that CBT/GET were successful because the outcomes were better!
So APT was an active intervention, and not a placebo control, and shouldn't be used as a control against which to compare CBT or GET.
A comparison with APT doesn't indicate the benefits of CBT or GET, but it illustrates the harm associated with their novel therapy, APT.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
I still don't know if there was a separate-analysis for various baseline variables (e.g. patients meeting the international criteria and patients with psychiatric comorbidity.) They do say that they factored baseline variables into the analysis, but I don't understand how this impacted the analysis, or if they actually did a separate sub-analysis for various baseline variables. They don't seem to have published any results from such an analysis, if they carried it out.

The only information I can find about it, is in the following extract, which I don't understand (If anyone can interpret it, could you let me know?):
If there are unmeasured variables that aff ect both mediator and outcome, the estimate obtained for this relation might be biased. Although unmeasured confounding cannot be ruled out, adding baseline variables that might be confounders makes a causal interpretation more plausible; further covariates were included in the models to address this.18,21 For example, if these variables were confounders, including them will have provided an adjusted and hopefully more accurate estimate, and if they were proxies for unmeasured confounders they might have partially adjusted for the omitted variables as well. The variables were selected for inclusion in models because they were thought to be potential predictors of mediators and outcomes. Age and sex could also be proxies for other variables that were not measured. The original trial stratifi cation factors were included to respect the trial design. The included baseline variables were: centre, Standardised Clinical Interview for DSMIV (SCID) depression status,22 London criteria for myalgic encephalomyelitis status,23 International Criteria for chronic fatigue syndrome status,1 baseline measure of mediator, baseline measures of both outcomes, baseline work and social adjustment scale,24 any anxiety disorder as determined using the SCID, age, sex, chronic fatigue syndrome patient group membership, receipt of fi nancial benefi ts, being in dispute regarding financial benefits, physical illness attribution, fibromyalgia status,25 illness duration, Jenkins Sleep Score, employment status, body-mass index, and physical symptoms (Patient Health Questionnaire-15) score.26 All these variables were included in all models, except where they were likely to be collinear with the baseline measure of the mediator (ie, SCID depression was omitted in the HADS depression model, and likewise for anxiety).
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
I've found some more quotes re the importance of cause and effect, and timing of changes. See the red text vs the final quote:

"Mediation is a hypothesised causal chain in which a baseline variable R affects a post-baseline mediating variable M, which in turn affects an outcome variable Y.18"

"The 12 week measure of the mediator was used to capture change as early as possible and have the maximum possible separation between mediator and outcome measurements. This temporal separation between the variables was employed to meet the implicit mediation model assumption of ordering of the variables in the causal chain.18 This ordering is important for rendering causal mediation inferences more plausible."

"Given the pattern of change in the mediators was similar to the pattern of change in the outcomes it is possible that the variables were affecting each other reciprocally and more measures might have helped clarify this."
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Mediating Effects.

Quote from paper:
"Fear avoidance beliefs, the strongest mediator, accounted for up to 60% of the overall effect, providing evidence for partial mediation. Many of the mediators accounted for much smaller proportions of the overall effects, suggesting that in some cases the effects of treatment on outcomes might have been mediated through several small effects and that some of the overall treatment effects were mediated through variables that were not measured."

I've had a look at the mediating effects in the appendix, and 60% was an unusually high score, even for fear-avoidance.
It was only as high as 60% when comparing GET with APT (61% for fatigue, 51% for SF-36 PF). When comparing GET with SMC, it was 17% for fatigue, and 43% for SF-36 PF. Still with fear-avoidance, CBTvsSMC was also 17% for fatigue, and 27% for SF-36 PF. Overall, mediating effects for fear-avoidance were between 17-61%. So it's a mixed bag, and not as clear-cut as I was led to believe from the abstract or the media. (i.e. 60% is pretty much an outlier.)

For the 6MWT, mediating effects were between 10-47%.

Below, I've posted the mediating effects for fear-avoidance and for the 6MWT. These two mediators have the highest effects. Most of the other mediator effects are between 0-20%.

Appendix document - Table E - Mediating effects.

% mediated
Fear avoidance - Fatigue
GETvsAPT 61%.
GETvsSMC 17%
CBTvsAPT 34%
CBTvsSMC 17%
Fear avoidance - SF-36 PF
GETvsAPT 51%.
GETvsSMC 43%
CBTvsAPT 37%
CBTvsSMC 27%

Many of the percentages are low e.g. between 0-20%
The next biggest percentages was for metres walked.

% mediated
Metres walked - Fatigue
GETvsAPT 33%
GETvsSMC 29%
CBTvsAPT 10%
CBTvsSMC 29%
Metres walked - SF-36 PF
GETvsAPT 34%.
GETvsSMC 47%
CBTvsAPT 13%
CBTvsSMC 21%
 
Last edited:

user9876

Senior Member
Messages
4,556
Some notes.

Analysis - individual or average outcomes?
Despite having read the whole paper, but not understanding all of it, I still don't know if the analysis is based on individual or average scores.

They use linear regressions over the data so the results are based on individual data vectors (which may be of interest to anyone doing FoI for data as it suggests that they have the data in a form with treatments, initial variables, mediator variables and outcome variables for each patient. And i think that the 6mwt is one of the mediators at 24 weeks.
 

user9876

Senior Member
Messages
4,556
I still don't know if there was a separate-analysis for various baseline variables (e.g. patients meeting the international criteria and patients with psychiatric comorbidity.) They do say that they factored baseline variables into the analysis, but I don't understand how this impacted the analysis, or if they actually did a separate sub-analysis for various baseline variables. They don't seem to have published any results from such an analysis, if they carried it out.

The only information I can find about it, is in the following extract, which I don't understand (If anyone can interpret it, could you let me know?):

As I understand it (and my understanding of the technique is dodgy) they compute two linear regressions,

mediator = a*treatment + a'*initial_condition_variables
outcome = b*mediator + b'*initial_conditions_variables + c*treatment

The treatment variable is either a 0 or 1 for the control or treatment.
They then multiply a and b together and see if it is a significant correlation. They talk about using a bootstrap method which I think is about testing the significance of the correlation but I'm not sure.

The thing I am not sure about is the a' and b' whether they join into the correlation or are ignored (ignored is my belief). I've seen a few papers where they suggest various models could be used rather than a simple linear model but as I understand it they have chosen a linear model - but I need to reread the paper after reading a bit about mediation techniques.

Using regression they of course have a number of assumptions including linear relationships between variables and (assuming least squared regression) particular error distributions for the residuals.

I need to reread the paper to be sure but I think to answer your question the initial variables are included in the model equations and hence they are included. On an initial reading I did find their paper a little vague with details but then I had no clue about what they were talking about.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Thanks for all of that, user.

I need to reread the paper to be sure but I think to answer your question the initial variables are included in the model equations and hence they are included.
Yes, it seems that they have included baseline variable in the analysis, but I don't understand to what purpose they did this.
Partly because I can't see that they've published any results based on different baseline variables.
And I can't see any discussion in relation to which baseline variables had an influence on results, or no influence.

Have you noticed any such results or discussion?