PACE Trial and PACE Trial Protocol

[Dolphin]

I suspect the reason Knoop used +/- 1SD is that he was looking at recovery, not norms. It would be hard to argue that the threshold for recovery included those between -1 & -2 SD (bottom 2.5%-16% of the population) since peope with CFS were unlikely to be there en masse before they got ill. More reasonable to set the recovery threshold a little higher, hence -1SD, though that's just speculation on my part.

As Biophile points out, SDs for a healthy population - which is more homogenous by definition - are smaller than for the full population. From what I've seen, the mean for SF36 PF is around 95for a working age healthy population with SD around 10, which would give >=75 as 'normal' and >=85 as recovered.Hope this helps.

Good points.

Personally, I think thresholds should be set individually: why should the threshold for recovery be the same for somebody who is 20 and somebody who is 60. Similarly, for some measures, I think thresholds should be broken down by gender. This process could either be automated using a program or else done individually, which again wouldn't take particularly long.

It may (or may not) be for this reason that the mean+(-)1SD method was used i.e. mean [mixed sample of all ages]-1SD is likely to be mean-2SD, for all the different means for different ages.

Personally, I don't like the mean+(-)1SD: (extending the point oceanblue made) a recovered group shouldn't be mostly or all "worse" than the mean: there should roughly the same better than normal as there are a bit worse than normal. The way to test this is to get the mean and standard deviation of the recovered group and see if it's significantly different from a healthy group. If it is, it suggests it's not a proper recovered group.

 

[Bob]

One observation: while using norm-based stats on non-normal population datashould be nonsense, as it happens the mean -1SD formula used by PACE sets the threshold at around the 16th percentile for the Bowling population data they used, pretty well where it would be if the data had been normally distributed. I don't know if this is chance or not, but it is worth noting.

I thought that they used the Health Survey for England for the normative data set.
Has anyone come to a different conclusion?

Has anyone managed to find out what the exact percentile is for an SF-36 PF score of 60 using any normative data set?

 

[oceanblue]

I thought that they used the Health Survey for England for the normative data set.
Has anyone come to a different conclusion?

Has anyone managed to find out what the exact percentile is for an SF-36 PF score of 60 using any normative data set?

From PACE:

[33] Bowling A, Bond M, Jenkinson C, Lamping DL. Short form 36
(SF-36) health survey questionnaire: which normative data should
be used? Comparisons between the norms provided by the
Omnibus Survey in Britain, The Health Survey for England and
the Oxford Healthy Life Survey. J Publ Health Med 1999, 21: 255–70

I thought they used the Omnibus survey, but not sure.
.

 

[Bob]

Thanks Ocean. I believe that they issued an informal amendment, to Hooper, which said they used data for 'English adults'. (Instead of 'UK working age population'.) So I think this would correspond to the Health Survey for England 1996. I've been assuming that's what they mean, anyway.

Edit: I've made a correction in this post: 'UK working age population'

 

[Esther12]

They (PACE) have to have used the PF data from table 3 in Bowling. It doesn't look like this is the same as the HSE 1996 data used in table 4. Was table 3 the result of the Bowling paper, rather than from any of the three data sets looked at in table 4? (Not long ago, I knew this stuff.)

 

[Bob]

They (PACE) have to have used the PF data from table 3 in Bowling. It doesn't look like this is the same as the HSE 1996 data used in table 4. Was table 3 the result of the Bowling paper, rather than from any of the three data sets looked at in table 4? (Not long ago, I knew this stuff.)

Oh, Esther, I'm just the same... I can know a subject inside-out one week, and then a few weeks later, I can't remember even the basics, and I can't remember what i've forgotten, which makes conversations about it hopeless!

I never knew which data sets they used from Bowling, so I can't answer your question.
I assumed that they could pick any of the data sets that Bowling discussed.

 

[Esther12]

I was trying to answer one of your questions Bob! This thread has been moving too rapidly for me to keep up recently though. Curses.

 

[Bob]

I was trying to answer one of your questions Bob! This thread has been moving too rapidly for me to keep up recently though. Curses.

Oh, sorry, I thought you were asking a question!
Table 3 relates to the British ONS Omnibus 1992 survey.
But that doesn't mean that they used that data set.
The paper discusses all three data sets, and highlights the pros and cons with each.
It doesn't come to an absolute conclusion about which is best, although it does suggest that postal questionairres are preferable (Only the Oxford Central England Healthy Life survey was postal.)
I think they used the HSE, because it's the lowest scoring of all three sets, and because they said they used an English adult population, after issuing an informal correction for the PACE paper.

 

[biophile]

Thanks Ocean. I believe that they issued an informal amendment, to Hooper, which said they used data for 'English adults'. (Instead of 'English working age adults'.) So I think this would correspond to the Health Survey for England 1996. I've been assuming that's what they mean, anyway.

I think the clarification regarding physical function norms was in the official authors reply in the Lancet, where they even defend the notion that using a general population was a good idea. The Bowling et al data was indeed "English adults" ie a general population, whereas in the Lancet paper they claimed to be using a working age population.

There has been a consistent pattern of goalpost shifting throughout the course of the trial, for example:

The early trial identifier only cites Jenkinson et al to justify mean-1SD=90(15)=75. The 2007 protocol cites both Jenkinson and Bowling to justify 85(15)=70 but they still use 75 as the threshold, probably because it is one step above 70 to be sure and clearly above the entry criteria of <=60. The 2011 Lancet paper only cites Bowling et al to justify 84(24)=60 but they not only mislabeled it a working age population but raise the entry criteria to <=65 to increase recruitment which is higher than their threshold for "normal"!

 

[Esther12]

Bob:

I see. Thanks - I'd forgotten where the data in table 3 came from.

I'm pretty sure that the data in table three (for the whole population, not just working age) gives the mean and SD used in PACE. That would be a strange coincidence if this was not the data that they used.

 

[Bob]

Bob:

I see. Thanks - I'd forgotten where the data in table 3 came from.

I'm pretty sure that the data in table three (for the whole population, not just working age) gives the mean and SD used in PACE. That would be a strange coincidence if this was not the data that they used.

No, it gives a mean of 89.6, the highest score of all SD 19.3
It gives that in table 4, under the 'ONS survey' column.
Look at page 266 for sample means for all the surveys.

 

[Bob]

I think the clarification regarding physical function norms was in the official authors reply in the Lancet, where they even defend the notion that using a general population was a good idea. The Bowling et al data was indeed "English adults" ie a general population, whereas in the Lancet paper they claimed to be using a working age population.

Thanks for that biophile... I'd forgotten it was a formally published correction.
"(in our paper we stated that this was a UK working-age population, whereas more accurately this should have been an English adult population)"
http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(11)60651-X/fulltext

 

[biophile]

Thanks biophile.

I have had a hard time keeping track of versions 1 & 2 of the London ME criteria, although have heard Ellen Goudsmit said they used version 2 not version 1.

Judging by what you wrote, I think this is version 1:
http://www.mesite.dk/LondoncriteriaUK.htm
and this is version 2:
http://phoenixrising.me/research-2/defining-chronic-fatigue-syndrome-mecfs/cfs-definitions-over-time

I should clarify that I got my information from Charles Shepherd's comments here (http://www.meassociation.org.uk/?p=4702) and have not read the entire discussion yet. There appears to be confusion about how many versions there are and what the citation in the Lancet paper actually refers to, so I may have been wrong about what should be called v1, but there are differences none the less.

So I need to do more reading from multiple sources I've identified, I wish the London ME criteria was published and accessible, it is ridiculous that 15 months after the Lancet publication all this still seems unresolved. Why does every aspect of the PACE Trial have to be a giant sinkhole?

When Hooper raised the issue, PACE simply replied that the London ME criteria was "handed" to them and they didn't "make up" their own. However, it still seems unclear who handed it to them and Hooper has asked for further clarification (without reply?). I think I've read that AfME may have been a source since they collaborated with PACE during the trial, but I'm still unclear. This all reminds me of the rumour that the individuals involved in helping PACE define APT and the 70% rule have vanished from AfME and are mysteriously untraceable. If that was true and I was a more suspicious person, I would suspect a setup.

White and Sharpe have, on one hand suggested that ME and CFS aren't necessarily the same, then on the other hand disingenuously claiming that PACE findings apply to both however it is defined. It is obvious that PACE has deviated away from the original descriptions of ME eg Ramsay, making it more like an Oxford criteria with a few additional symptoms. They weren't interested in defining an original ME cohort, only one that doesn't interfere with Oxford.

If PACE wanted to compare CFS and CBT/GET with ME and pacing more seriously, they should have engaged known professional proponents. Obviously the Canadian criteria was "impossible" for them to use because it interfered with their concept of CFS, not because they couldn't implement it if they wanted to.

 

[Esther12]

No, it gives a mean of 89.6, the highest score of all SD 19.3
It gives that in table 4, under the 'ONS survey' column.
Look at page 266 for sample means for all the surveys.

Sorry if I've made an error based on forgetfulness here... but this doesn't seem right. Look at the figures in table 3 - they're not the same as those in table 4. I think I (and others) manually added up and calculated the mean and SD for the PF figures in table 3, and they gave the PACE figures.

 

[Bob]

Sorry if I've made an error based on forgetfulness here... but this doesn't seem right. Look at the figures in table 3 - they're not the same as those in table 4. I think I (and others) manually added up and calculated the mean and SD for the PF figures in table 3, and they gave the PACE figures.

Don't worry about it Esther. I don't know the answer - I'm just guessing, based on what they've said.
I might be wrong, but I'm pretty sure that they just picked the lowest scores possible from the Bowling paper.
I've never worked out how they got '84' though, as it doesn't match any of the means.
It is the mean for all men though, using the HSE, so I don't know if maybe they just made another mistake there!
I'd be very surprised if they adjusted the mean to match for the age, gender, social class etc., of the Trial patient sample - because I can't imagine them going to that much trouble!
Table 3 and 4 are different because table 4 splits the ages up between sexes.

 

[Sam Carter]

Don't worry about it Esther. I don't know the answer - I'm just guessing, based on what they've said.
I might be wrong, but I'm pretty sure that they just picked the lowest scores possible from the Bowling paper.
I've never worked out how they got '84' though, as it doesn't match any of the means.
It is the mean for all men though, using the HSE, so I don't know if maybe they just made another mistake there!
I'd be very surprised if they adjusted the mean to match for the age, gender, social class etc., of the Trial patient sample - because I can't imagine them going to that much trouble!
Table 3 and 4 are different because table 4 splits the ages up between sexes.

Hi Bob,

I think they might have derived the normal range for SF-36 PF sub-scale scores by taking the average of these figures from Table 3 in Bowling et al:

Physical Functioning

................Mean.....(SD)
Sex
Male........86.3.......(22.5)
Female....81.3.......(25.7)

This would give an (average) mean of 84.05 and an (average) SD of 24.1, as reported in The Lancet, but I'm really just guessing. (I'm pretty sure they were too.....)

 

[Esther12]

Table 3 and 4 are different because table 4 splits the ages up between sexes.

These are my workings for table 3:

Mean (using the means from both sexes): [(86.3*925)+(81.8*1117)]/(925+1117)=83.83

And the SD for both sexes: [(22.5*925)+(25.7*1117)]/2042=24.25

These look like the figures used by PACE.

I get a slightly different mean and SD when using the data from all the different age ranges (103 people were not included in male/female data):

Mean:

[(95.5*204)+(94.5*415)+(93.3*319)+(87.2*297)+(78.0*297)+(72.7*281)+(57.9*296)+(39.3*36)]/(2145)= 82.288...

SD:

[(12.1*204)+(13.5*415)+(13.4*319)+(20.9*297)+(26.3*297)+(26.7*281)+(28.6*296)+(31.5*36)]/2145=20.26

So the 'normal range' for the entire population (a crazy measure for assessing those of working age) which was chosen by the PACE researcher really starts at 62. Another little error.

I think that the PACE trial used the data that was broken down in to 'male' and 'female'.

edit: Sam got the same point across without quite so much tedious mathematics. Ah well... good to give my brain a little work out.

 

[Esther12]

Actually... the move from 60 to 62 could have been pretty significant, because of the way the SF36 PF is scored. Ah well.

 

[Bob]

Hi Bob,

I think they might have derived the normal range for SF-36 PF sub-scale scores by taking the average of these figures from Table 3 in Bowling et al:

Physical Functioning

................Mean.....(SD)
Sex
Male........86.3.......(22.5)
Female....81.3.......(25.7)

This would give an (average) mean of 84.05 and an (average) SD of 24.1, as reported in The Lancet, but I'm really just guessing. (I'm pretty sure they were too.....)

Ha! Yes, Sam, I'm convinced that they were guessing! They don't seem to have a clue!
But thanks for that... It could explain it... Except that they have said they used the 'English adult population'... And the total sample mean is given as 89.6 for the ONS data.
I don't think we'll ever know, unless someone does a FOI request.

 

[Bob]

These are my workings for table 3:

Mean for all (using the means from both sexes): [(86.3*925)+(81.8*1117)]/(925+1117)=83.83

And the SD for both sexes: [(22.5*925)+(25.7*1117)]/2042=24.25

These look like the figures used by PACE.

I get a slightly different mean and SD when using the data from all the different age ranges (103 people were not included in male/female data):

Mean:

[(95.5*204)+(94.5*415)+(93.3*319)+(87.2*297)+(78.0*297)+(72.7*281)+(57.9*296)+(39.3*36)]/(2145)= 82.288...

SD:

[(12.1*204)+(13.5*415)+(13.4*319)+(20.9*297)+(26.3*297)+(26.7*281)+(28.6*296)+(31.5*36)]/2145=20.26

So the 'normal range' for the entire population (a crazy measure for assessing those of working age) which was chosen by the PACE researcher really starts at 62. Another little error.

I think that the PACE trial used the data that was broken down in to 'male' and 'female'.

edit: Sam got the same point across without quite so much tedious mathematics. Ah well... good to give my brain a little work out.

Ah, OK, maybe that could explain it. Thanks for that Esther.
It still doesn't explain their correction though, does it.
Basically, they made it all up, because they didn't have a clue what they were doing!