• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

BMJ comments on new PACE trial data analysis

user9876

Senior Member
Messages
4,556
I actually disagree, respectfully, and only to try to sharpen the case that needs to be made, with both these analyses and think Dr Chalder would be correct to challenge them.

All symptoms are subjective. Even if you have a broken leg. Fluge and Mella have used secondary physical measures but these will have no statistical validity.

Are you making the point that there is no statistical validity in using secondary measures as it is too easy to just pick those that fit your hypothesis. I was wondering if the real issue is with the way significance testing is done - why do medical researchers not use a Bayesian approach to evidence and try to estimate the probability of a hypothesis given all the evidence and the probability of not the hypothesis given all the evidence.
 

MeSci

ME/CFS since 1995; activity level 6?
Messages
8,231
Location
Cornwall, UK
I too would like to know what "specialist medical care" in the PACE trial meant. Probably little more than nothing.

There is a brief description on page 3 here.

Specialist medical care (SMC)
SMC was provided by doctors with specialist experience in
chronic fatigue syndrome (webappendix p 1). All participants
were given a leaflet explaining the illness and the nature of
this treatment. The manual was consistent with good medical
practice, as presently recommended.2 Treatment consisted of
an explanation of chronic fatigue syndrome, generic advice,
such as to avoid extremes of activity and rest, specific
advice on self-help, according to the particular approach
chosen by the participant (if receiving SMC alone), and
symptomatic pharmacotherapy (especially for insomnia,
pain, and mood).
 

Dolphin

Senior Member
Messages
17,567
Can anyone give me some examples of the kind of specialist medical care ME/CFS patients in the UK are supposed to be routinely offered? When Chalder et al. say that "adding cognitive behaviour therapy (CBT) and graded exercise therapy (GET ) to specialist medical care (SMC) both had greater success in reducing fatigue and physical disability than adding adaptive pacing therapy (APT) to specialist medical care (SMC) or SMC alone", what SMC do they have in mind?
I've never been offered any medical care, specialist or otherwise, for my ME/CFS apart from GET/CBT.
One can read the SMC treatment manual here: http://www.pacetrial.org/docs/ssmc-doctor-manual.pdf
 

Marco

Grrrrrrr!
Messages
2,386
Location
Near Cognac, France
SMC? I always laughed at that. 'Specialist' for me involved being moved from pillar to post to investigate various symptoms (and then ignore the results). Decades later I was referred to the 'Chronic Fatigue Clinic at the Belfast City Hospital that turned out to be a screening service to ensure that you didn't have any 'serious' disease like diabetes, heart disease or cancer that causes fatigue (not a bad idea). I believe it's now closed. Thereafter I was discharged back to the useless GP as 'confirmed' (sic) CFS.

I have to admit that I was concerned when the Ritux trials seemed to be using the same symptom based subjective measures as a primary outcome as PACE. But i tend to agree with Jonathan that short of hard physiological measures that 'objective' outcomes such as metres walked, actigraphy, employment etc are subject to too many variables.

My own N=1 subjective experience backs thiis up. I struggled on for 20 years through university and full time work with great difficulty until I collapsed ten years ago and was subsequently medically retired. I could say that my ability to work correlated with the severity of my symptoms but I'm now probably back to where I was prior to medical retirement. So could I work now? I'd say to the contrary that for 7 years prior to being medically retired that I really wasn't up to working. But what was the alternative? I just pushed on every day and didn't take sick days as there wasn't any point. I knew that I'd feel ,just as bad the day after a 'sick day'. Eventually it just ground me down.

Sorry for the rant.
 

Jonathan Edwards

"Gibberish"
Messages
5,256
Are you making the point that there is no statistical validity in using secondary measures as it is too easy to just pick those that fit your hypothesis. I was wondering if the real issue is with the way significance testing is done - why do medical researchers not use a Bayesian approach to evidence and try to estimate the probability of a hypothesis given all the evidence and the probability of not the hypothesis given all the evidence.

Normally secondary measures are sunk by the Bonferoni correction with marginal power in the first place. If you do statistics on the secondary measures you compromise the validity of your primary analysis too. I think the hypothesis driven approach to statistics may run into too many unascertainables about the certainty of a particular prediction applying to a particular hypothesis and how wide a net a hypothesis can cast.
 

Sasha

Fine, thank you
Messages
17,863
Location
UK
In terms of objective vs subjective assessments of walking (as opposed to 'feeling ill'), I think there's a lot to be said for actigraphy. I think it's hard to remember what your baseline of activity was several months ago and to try to judge whether you've improved, got worse or got better (bearing in mind that any changes are probably going to be fairly small for most patients).
 

Dolphin

Senior Member
Messages
17,567
In terms of objective vs subjective assessments of walking (as opposed to 'feeling ill'), I think there's a lot to be said for actigraphy. I think it's hard to remember what your baseline of activity was several months ago and to try to judge whether you've improved, got worse or got better (bearing in mind that any changes are probably going to be fairly small for most patients).
(As many people will know) In the PACE Trial, they found actigraphy useful enough to measure for seven days before the therapies. It was after the treatment was over that they felt it shouldn't be used. o_O
 
Last edited:

Jonathan Edwards

"Gibberish"
Messages
5,256
In terms of objective vs subjective assessments of walking (as opposed to 'feeling ill'), I think there's a lot to be said for actigraphy. I think it's hard to remember what your baseline of activity was several months ago and to try to judge whether you've improved, got worse or got better (bearing in mind that any changes are probably going to be fairly small for most patients).

Yes, I'd buy that. Maybe for some people the assessor would do a pre-trial screen and then sit down together with the person and come up with - 'well if I could do 250 blips instead of 60 I think I would rate that a grade 1'. And maybe for others they might agree 'well I go up and down so much from day to day I don't think this would be my priority measure'. You can still measure everyone and analyse but you can also keep the personalised endpoint clean for the primary statistical analysis.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
To reflect reality, participants receiving specialist medical care should have been greeted with a disbelieving and impatient expression, or zero eye contact, a grunt and a shrug, told there was nothing wrong with them, then eventually sent off to receive CBT. But perhaps that wouldn't be ethical. I like the way the SMC group were given a leaflet, as almost the full extent of their 'specialist' treatment. I was going to say that SMC should have been called bog-standard-care but, to be fair, you don't get given a leaflet in bog-standard-care. (Well, I never received a leaflet when I was diagnosed, but I diagnosed myself, so it seemed a bit silly to give myself a leaflet.)
 
Last edited:

Tom Kindlon

Senior Member
Messages
1,734
Another substantial BMJ e-letter on the UK£5 million PACE Trial, this time from Alem Matthees.

He highlights various changes to the protocol.

He also highlights how the recovery criteria are much easier than planned (and there was an error in the justification given for changing them).

http://www.bmj.com/content/350/bmj.h227/rr-16
Erica Verrillo has done a new (Jan 30) piece on this:
Australian Researcher Challenges Measures of "Recovery" in PACE Trial:
http://cfstreatment.blogspot.ie/201...challenges.html#sthash.bmmt0iZc.e2e3nZ3D.dpuf
 

ahimsa

ahimsa_pdx on twitter
Messages
1,921
I think it's hard to remember what your baseline of activity was several months ago and to try to judge whether you've improved, got worse or got better ...

This is a key problem for questionnaires. And not just for ME/CFS patients, for anyone! Even if a person is trying very hard to be truthful and objective it is difficult to accurately list, for example, just what you had to eat yesterday (e.g., those studies trying to link diet to disease). It's even harder to remember how symptoms have changed over time.

I will often forget a key symptom when talking about my illness. The most blatant example for me (that I can remember) is the time when I was in my doctor's office with my husband. I was reporting my symptoms and I said something like, "I haven't had a migraine in weeks." And I really believed that. Then my husband said, "But you just had one on Tuesday." I was embarrassed (which is probably why I still remember this incident) but it was true. I had just had a migraine a few days earlier.

I was not trying to be super positive about my symptoms because I was listing a bunch of other problems. And I don't think I was subconsciously omitting information to make the doctor happy. I had simply forgotten all about it. As soon as my husband mentioned it then I remembered it. So that memory was in my brain somewhere. And yet, when asked to list my symptoms, I had completely forgotten one of the big ones that had just happened recently and wiped me out for half a day.

This kind of thing happens to me all the time. It's so frustrating because I did not used to be so ditzy and flakey (former software engineer, attention to detail, etc.). I actually had a lot less brain fog at the start of my illness. Not sure why.

I know that no one has a perfect memory, and has biases, and so on. But this is ridiculous.

I'm not saying don't use questionnaires for ME/CFS studies, or don't believe ME/CFS patients, or anything like that. I'm just pointing out a problem (which I'm sure is obvious to most of you) with using questionnaires.

Hope this is useful and not a derail.
 

Countrygirl

Senior Member
Messages
5,429
Location
UK
Here is a response by patient Sheila Campbell to Dr TC: -


http://www.bmj.com/content/350/bmj.h227/rr-26
  • I would like to draw your attention to an error in the rapid response from Chalder et al (1), most of whom were also authors of the original PACE Trial paper. They wrongly stated that PACE was a “randomised, controlled trial”.

    The title of the PACE Trial, “Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial” clearly and correctly describes it as just a “randomised trial” not as a "randomised, controlled trial" (2)

    This is reiterated under “Method, study design” where it states “PACE was a parallel, four group, multicentre, randomised trial”(2).

    As has been explained here in several of the rapid responses, in common with most cognitive behavioural therapy research, the PACE trial failed methodologically to attain the gold standard of a randomised, controlled trial (3-5).

    For example, Robert Courtney points out:
    “Although it was a large and expensive government-funded trial, the PACE trial, as with most cognitive-behavioural research, was open-label and failed to control for placebo effects and biases such as response bias [24,25]. CBT and GET changed the way that a minority of patients interpreted their illness and responded to self-report questionnaires, as demonstrated by the 11-15% self-report clinical response rate to CBT/GET, but as placebo effects and response bias were not controlled for in this open-label study, it is possible that the self-reported effects could be explained by weaknesses of the trial methodology [24-28].” (3)

    May I suggest this significant error is corrected to ensure accuracy and avoid further confusion.

    References:
 
Last edited:

Bob

Senior Member
Messages
16,455
Location
England (south coast)
In terms of objective vs subjective assessments of walking (as opposed to 'feeling ill'), I think there's a lot to be said for actigraphy. I think it's hard to remember what your baseline of activity was several months ago and to try to judge whether you've improved, got worse or got better (bearing in mind that any changes are probably going to be fairly small for most patients).
Yes, the Chalder fatigue scale requires you to compare yourself to how you 'normally' feel (e.g. "Do you have problems with tiredness?" "Yes, more than normal"), which I think is supposed to mean how you used to feel before becoming ill, but it could be interpreted to mean pre-illness, or baseline illness, or whatever you consider to be normal levels of symptoms. I'm sure participants must have got confused and answered with a great deal of variability. After ten or so years of illness, feeling run-down becomes 'normal' and pre-illness good-health is a weird concept that's difficult to remember. So these questions can't be answered with any consistency. Not to mention the ceiling effect whereby many ME patients score near the maximum score on the Chalder scale which means that they can't statistically deteriorate during the trial, but can only improve!
 
Last edited:

Dolphin

Senior Member
Messages
17,567
One can improve, at least somewhat, the accuracy of self-reporting by using live, ecological measurements e.g. electronic diaries. These involve getting a prompt asking for the level of one or more particular symptoms at a particular moment in time. Fred Friedberg had tested this in at least one CFS study:
Friedberg F, Sohl SJ. Memory for fatigue in chronic fatigue syndrome: the relation between weekly recall and momentary ratings. Int J Behav Med. 2008 Jan-Mar;15(1):29-33.
 
Last edited:

Sasha

Fine, thank you
Messages
17,863
Location
UK
Yes, I'd buy that. Maybe for some people the assessor would do a pre-trial screen and then sit down together with the person and come up with - 'well if I could do 250 blips instead of 60 I think I would rate that a grade 1'. And maybe for others they might agree 'well I go up and down so much from day to day I don't think this would be my priority measure'. You can still measure everyone and analyse but you can also keep the personalised endpoint clean for the primary statistical analysis.

I'm a bit confused about this grades idea. I think that PWME's activity levels would be on a continuous scale, not in big jumps like this. Why not measure blips instead of grades, so to speak?

The grades are an outcome measure, not GET goals?
 

Dolphin

Senior Member
Messages
17,567
Yes, the Chalder fatigue scale requires you to compare yourself to how you 'normally' feel (e.g. "Do you have problems with tiredness?" "Yes, more than normal"), which I think is supposed to mean how you used to feel before becoming ill, but it could be interpreted to mean pre-illness, or baseline illness, or what you consider to be normal levels of symptoms. I'm sure participants must have got confused and answered with a great deal of variability. But after ten or so years of illness, feeling run-down becomes 'normal' and pre-illness good-health is a weird concept that's difficult to remember. So these questions can't be answered with any consistency. Not to mention the ceiling effect whereby many ME patients score near the maximum score on the Chalder scale which means that they can't statistically deteriorate during the trial, but can only improve!
(As some people will know)

In this MS trial, which used a similar type of CBT to that used in the PACE Trial (note Trudie Chalder was involved and I have read up on this) they measured fatigue using the Chalder fatigue scale:
Psychosom Med. 2008 Feb;70(2):205-13. doi: 10.1097/PSY.0b013e3181643065. Epub 2008 Feb 6.
A randomized controlled trial of cognitive behavior therapy for multiple sclerosis fatigue.
van Kessel K1, Moss-Morris R, Willoughby E, Chalder T, Johnson MH, Robinson E.
After CBT, people with MS rated their fatigue as an average of 7.90! That's a lot better than healthy people (who score an average of 11.2). The default score if you have no problems with fatigue is 11.
 

Jonathan Edwards

"Gibberish"
Messages
5,256
I'm a bit confused about this grades idea. I think that PWME's activity levels would be on a continuous scale, not in big jumps like this. Why not measure blips instead of grades, so to speak?

The grades are an outcome measure, not GET goals?

If you personalise targets for criteria you have to use a non-numeric discontinuous score. That has statistical costs but since there is little reason to think that most continuous scales that would be relevant could be treated in a parametric way the cost may be not so great. The advantage is that you can get something that reflect more realistically people's health goals - complete recovery, substantial improvement, no change, worse. It makes it difficult to sell a small quantitative difference as a recommended treatment when in fact the effect is neither here nor there for the patient. It is probably only of value if you are looking for pretty substantial effects, even if only in a small proportion. You can have a near continuous range of intermediate grades without major problems if you want to but it is likely to become unwieldy setting up all the individual scales.
 

Jonathan Edwards

"Gibberish"
Messages
5,256
(As some people will know)

In this MS trial, which used a similar type of CBT to that used in the PACE Trial (note Trudie Chalder was involved and I have read up on this) they measured fatigue using the Chalder fatigue scale:

After CBT, people with MS rated their fatigue as an average of 7.90! That's a lot better than healthy people (who score an average of 11.2). The default score if you have no problems with fatigue is 11.

That seems very telling - although what it is telling they may not be telling.

Reminds me of the old saying:
You can always tell a Bart's man;
But you can never tell him anything.