1. Patients launch $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
ME/CFS and the Magic of the Canine Factor
There's been plenty of research indicating that having pets is good for your health. I never really noticed any particular benefits to having cats, though that may have had more to do with my cats. They've been fairly indifferent to my presence and we've shared a live-and-let-live...
Discuss the article on the Forums.

PACE Trial and PACE Trial Protocol

Discussion in 'Latest ME/CFS Research' started by Dolphin, May 12, 2010.

  1. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    343
    UK
    Peter White in 2000 on subjective (SF-36 PF) vs objective measures

    Just been reading a paper looking at the exercise capacity of CFS patients, which I will post on later. But for now I was struck by the authors' comments on the value of objective over subjective measures:

    Strength and physiological response to exercise in patients with chronic fatigue syndrome. 2000.
    Kathy Y Fulcher, Peter D White

    Talking about the SF-36 Physical Function scores. (my italics):
    Talking about the results of their GET programme:
    You might have thought that would lead Peter White to not rely solely on subjective measures as the primary outcome of PACE.

    Dolphin: I think there is a general problem inusing questionnaires, tests (or even actometers)that no one really measures their ability to accurately measure small changes. The only such testing I've come across looks at changes in the test/score correlated with eg hip replacement surgery, where a large change is expected.
    biophile, WillowJ and Dolphin like this.
  2. Dolphin

    Dolphin Senior Member

    Messages:
    6,698
    Likes:
    5,503
    I recall Esther Crawley giving the SF-36 PF scores like this (10-30). Perhaps also in the odd other paper.

    So with SF-36 PF, one actually needs to ask oneself are they:
    (i) 0-100 (non-normed scores)
    (ii) normed scores
    or
    (iii) 10-30 scores.

    As if things weren't complicated enough for everyone in the ME/CFS field!
    biophile and oceanblue like this.
  3. Dolphin

    Dolphin Senior Member

    Messages:
    6,698
    Likes:
    5,503
    Almost all the reason for the difference is that the ME criteria were operationalised so that if the patients had a psychiatric disorder, they couldn't have M.E.

    If one just looks at the Oxford Criteria patients who didn't have a psychiatric disorder, 97% were classed as having M.E.

    To me, this is too high a figure.

    Below are the details on this calculation:
    ----------------
    In the Lancet paper on the PACE Trial (by White et al (2011)), it said:
    ----------

    -----------

    I was not 100% sure what "no primary depressive or anxiety disorder (interpreted as an absence of any such disorder)" meant in relation to the percentage we were given for "any psychiatric disorder" i.e. could there be
    an overlap.

    The following is an extract of a letter that clarifies it (see asterisked bit) - the letter was written by PD White, KA Goldsmith, AL Johnson, R Walwyn, HL Baber, T Chalder, M Sharpe, on behalf of all the co-authors (of the PACE Trial)

    ---------
    The trial did not study ME/CFS (pages 12-18)



    (source: http://www.meactionuk.org.uk/whitereply.htm )

    -------
    The figures from the paper (Table 1) show that, of the 640 participants, 300
    (46.9%) had "any psychiatric disorder" and 51.4% (329) satisfied the London
    criteria*.

    This means that of the 340 patients in the trial (i.e. who satisfied the
    Oxford criteria**, with a principal symptom of fatigue, who did not have a
    psychiatric disorder), 329 (96.8%) satisfied the London criteria for M.E.,
    the definition of M.E. used in the trial!

    This is an amazingly high figure for a definition of M.E. given the
    "looseness" of the Oxford criteria e.g. unlike the Fukuda CFS criteria or
    Carruthers ME/CFS criteria, it doesn't require other symptoms apart from
    fatigue.

    It seems to me the definition for M.E., at least as it was used in this
    trial, is very suspect. And hence it is questionable what can read from
    into how people with M.E. responded in the trial.

    --------

    ----------
    *One can how the London Criteria were used here:
    London_Criteria_in_PACE_Trial.jpg

    **
    One can how the Oxford Criteria were used here:
    Oxford_Criteria_in_PACE_Trial.jpg

    Annex 1:
    We already cannot be sure if the participants are representative of patients
    in the community based on who refused to take part, were excluded, etc.
    oceanblue and Valentijn like this.
  4. Valentijn

    Valentijn Activity Level: 3

    Messages:
    6,250
    Likes:
    8,903
    Amersfoort, Netherlands
    If I understand correctly what you've said, 100% of participants had Oxford CFS. Approximately half of those also qualified as having the PACE trial's version of ME, but quite a few were likely excluded from having ME due to secondary psychiatric issues.

    So a larger proportion than half of the participants probably had ME, but were excluded from being labelled as such due to the manipulation of the ME definition used?

    Any idea why they would do this? The lower number of reported ME participants would seem to weaken their push to have CBT/GET applied to ME patients, which sounds counter-productive from their perspective. Not that anything they do makes much sense really :p
  5. Dolphin

    Dolphin Senior Member

    Messages:
    6,698
    Likes:
    5,503
    The graphs they used claim to show the response was similar in the "M.E." group.

    My point was more not focusing on the psychiatric exclusion point but the questions used to define M.E. I'm not sure whether the London criteria have the potential to be good or not, but I think there is a problem with how they defined M.E. in this study (i.e. maybe another wording might not be so bad). I think it's unlikely 97% of those who have Oxford criteria CFS (but don't have a psychiatric disorder) have M.E.
  6. Bob

    Bob

    Messages:
    8,048
    Likes:
    10,297
    England, UK
    I'm a bit slow to catch on sometimes... But something I've just noticed is...

    The figures say that 15% and 13% returned to 'normal' (as per the flawed 'normal range' analysis) as a result of CBT and GET.
    But only 14% and 16% 'improved' as a result of CBT and GET. (as per the post-hoc analysis).

    So for GET, more participants returned to 'normal' than 'responded' to treatment.

    School-boy error?
  7. Dolphin

    Dolphin Senior Member

    Messages:
    6,698
    Likes:
    5,503
    No, there is not necessarily any error. For example, the exact percentages that changed from baseline might be different: those figures for "return to normal" are only estimates - some people may have already been "normal" so there was no change.
  8. Esther12

    Esther12 Senior Member

    Messages:
    5,254
    Likes:
    5,432
    @ bob: Those improved figures look low. Didn't they re-define 'improved' in a way that allowed them to say 60% had a clinically significant improvement? I am forgetting this stuff, but I'm pretty sure those figures are wrong (or are they accounting for SMC in a way I've not seen before?)

    As a side point, because their definition of 'back to normal', overlapped with their definition of 'severe and disabling fatigue' it would be possible to a patient to have gotten worse, yet still be counted as having got back to normal following treatment.
  9. Mark

    Mark Acting CEO

    Messages:
    4,527
    Likes:
    2,004
    Sofa, UK
    Access to PACE Trial Data

    In my day job, I'm just starting work on a research data IT project, and along the way I've been reading about open data principles of UK funders. What I've been finding suggests some possibilities that may be useable to obtain access to the PACE trial raw data. Freedom of Information Requests for this information, and a parliamentary question, have so far failed to obtain a stated commitment to provide the data, but the information below suggests to me that the response of the researchers is not consistent with the Open Data principles of the funders which they have signed up to.

    (Quotes below with my emphasis)

    Research Councils UK (RCUK) General Principles


    Medical Research Council (MRC) Research Funding Policies

    Note: The excuse for not providing data in response to the FOI request was plans for future publication. The MRC's policy states that this must apply only for a limited defined period. If no date has been set for the release of the data then this is in contradiction to the MRC's policy. If no strategy for eventual sharing of the data was submitted at the proposal stage, this is also in contradiction to the MRC's policy.

    MRC Data-Sharing Policy:


    Freedom of Information and Research Data

    Access to the underlying data from the PACE trial is a crucial objective, and it is needed in time for it to be analysed in order to present a case for the NICE Guidelines Review.

    An FOI Request and parliamentary request have been submitted, but it is critical that these are followed up on, and that the researchers are compelled to fulfil their legal obligations and the obligations imposed by their funders.

    It appears that they have not complied with crucial sections of the policies and rules highlighted above.

    In particular:

    - The funders (the MRC) should be made aware that the researchers have failed to provide access to the research data in a timely manner, and should be asked to ensure that they comply with MRC policy.

    - There is a requirement from the MRC that, when research data is withheld, this must be for a defined period, and the data-sharing strategy must be specified in the funding proposal. Thus there is an obligation on the researchers to specify a date when the research data will be made publicly available.

    - The MRC stipulate that the researchers must maximise the opportunities for data-sharing, and their access policy must be transparent. The MRC should be approached to request that the researchers comply with these mandatory requirements.

    - The exemption used as a reason not to supply the data in response to the Freedom of Information Request does not apply unless the researchers have a stated plan to publish the information requested - not just a paper based on the results. If I understand correctly, the researchers have refused the FOI request on the basis that they are publishing a future paper based on the data. This is not sufficient for this clause: they must state their plan for releasing the underlying data. If this is correct, then the rejection of the FOI request should be appealed, with a demand that a date must be set when the underlying data requested will be revealed.

    In summary:
    It is a clear requirement of both the funders (MRC) and of FOI legislation that the researchers must state when they are going to release the underlying data, they must do so in a timely fashion, and future publication of derived research is not an excuse to avoid publication of the underlying data to a specified timescale. So far, the researchers appear to have succeeded in fobbing off requests for the data. It should not be a surprise that the initial answers to questions like these meets with rejection. It is essential, however, that if answers to these questions are genuinely sought - and they are crucial and urgent questions - they must be followed up with further pressure, and with reference to the legal requirements and the requirements of the funding bodies.

    The obligations of the researchers are clear: they need to be held to those obligations, and this is a matter of urgency in preparation for the NICE Guidelines Review.
    oceanblue, Enid and alex3619 like this.
  10. Bob

    Bob

    Messages:
    8,048
    Likes:
    10,297
    England, UK
    My point was that the Normal Range analysis was flawed, and maybe they should have noticed this because, according to their figures, more people returned to 'normal' than 'improved'. :confused:
    oceanblue likes this.
  11. Dolphin

    Dolphin Senior Member

    Messages:
    6,698
    Likes:
    5,503
    Ok, maybe that would prompt some people to question the definition of "normal", alright.
  12. Bob

    Bob

    Messages:
    8,048
    Likes:
    10,297
    England, UK
    Yes, they are exactly correct Esther... Exactly as presented in the Lancet paper, once the spin has been unspun.

    Only 13% improved as a result of CBT or GET. And CBT was clinically ineffective, for improving physical function. Neither of which we would know without analysing the data tables. In fact, throughout the paper they say that CBT was 'moderately effective' without qualification. (eg: "We suggest that these findings show that either CBT or GET, when added to SMC, is an effective treatment for chronic fatigue syndrome, and that the size of this effect is moderate”.) This misrepresents the data, because CBT was only moderately effective for one of the two primary outcomes. So we could equally say that CBT was ineffective, using their methodology.

    (If you add on the improvements seen in the SMC control group then you can add on another 58% or 65% for the improvement rates, but the improvements attributable to CBT and GET only, are as I've described.)

    The actual results are very different to the spun results, aren't they!
    WillowJ and Dolphin like this.
  13. Bob

    Bob

    Messages:
    8,048
    Likes:
    10,297
    England, UK
    Yes, that's what I getting at. I think that most people would assume that 'returning to normal health' would involve a higher degree of improvement than a minimum improvement. And it perhaps should have made the Lancet and the authors double check their figures. Then they might have noticed that the 'normal range' was a flawed analysis.
  14. Bob

    Bob

    Messages:
    8,048
    Likes:
    10,297
    England, UK
    Good thinking Mark.

    Coincidently, I'd had a look at this recently as well, and found the same info...

    The MRC's guide to the 'principles' of access to data states:

    "MRC research data are publicly-funded and, as a public good, must be made available for new research purposes in a timely, responsible manner."

    That's from a PDF file dated May 2007:
    Principles for access to, and use of, MRC funded research data
    http://www.mrc.ac.uk/Utilities/Documentrecord/index.htm?d=MRC003759

    Considering the date, do you think it might apply to the PACE Trial?

    It's definitely worth taking up with the MRC.
    Ideally, we need all of the raw data.
  15. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    343
    UK
    Great work, Mark.

    As the PACE proposal was approved pre-2005, I'm not sure to what extent, if any, PACE is bound by the 2005 guidelines. Would be good to know, and is worth following up. Any takers?

    I also think the MRC rules apply to access by other researchers, and probably wouldn't apply to access bypatients who are not formal researchers.

    As the PACE trial was based at Barts, I'm not sure Scottish FOI rules would apply so the general exemption still stands, as far as I can see, but I may have missed some of your points along the way. If so, apologies.

    Also, I heard somewhere that further PACE papers are being drafted (no timetable for publication) so there may be some action in the pipeline. This may provide sufficient defence for now but a defined timescale would surely be reasonable.

    and that is a very strong argument too.I suspect that PACE are planning to sneak something out at the last possible moment and as long as possible after the original paper came out (and media interest petered out too).
  16. Mark

    Mark Acting CEO

    Messages:
    4,527
    Likes:
    2,004
    Sofa, UK
    This is a key question: the application pre-2005 would have been subject to less stringent requirements, but the MRC will presumably desire that research that has received ongoing funding since that date complies with the latest best practice. They have had successful funding applications since 2005, I believe.

    The first job here, I think, is to draft a first letter to the MRC to query these points.

    That's not my understanding of Open Data. It's not stated anywhere that I've seen that the repositories should be accessible to researchers only. I think "Open Data" really does mean "Open Data"...

    You're right that the Scottish 12-week period on the exemption does not apply, but the FOI exemption only applies to data that is going to be published eventually. It's not enough to say that they are going to publish a paper based on that data, they must state when they are going to publish the data itself. It appears they have misapplied that exemption clause. That is a key issue to explore and perhaps challenge, maybe legally, because this FOI clause applies regardless of the date of commencement being pre-2005.

    A defined timetable appears to be mandatory in response to FOI requests, and mandatory for MRC-funded research. The MRC should be expected to apply this requirement even in respect of research that began before 2005. Having further papers in the pipeline is not an acceptable defence unless those papers promise to publish all the raw data. They must publish the raw data in a timely fashion, in response to FOI requests and as a requirement of MRC funding, and they must state when they are going to do this.

    I am quite sure that they will resist publishing the raw data for as long as they are allowed to do so. Unless the correct kinds of pressure are applied, they will do as little as they have to do. But I think that pressure via the MRC and FOI requirements ought to be able to get them to publish it much sooner than they wish.
  17. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    343
    UK
    Great plan
    I was basing that on this from your earlier post (and I think something I've read elsewhere too):
    I assumed 'eventual patient benefit' means benefit from other research with the released data.

    Could you just highlight the FOI clause in question? I'm phasing out here.

    That would be good.
  18. Mark

    Mark Acting CEO

    Messages:
    4,527
    Likes:
    2,004
    Sofa, UK
    My analysis was based on the guidance from JISC, see Question 12 here:
    http://www.jisc.ac.uk/publications/programmerelated/2010/foiresearchdata.aspx

    The answer to question 12 states:

    My reading of this is that, if they are asked for data, and claim exemption on the basis that the data is going to be published in future, they must have a written plan to release precisely the data requested, and not an analysis based on that data, and I think that implies that they must say when they are going to release it.

    A request for the entire raw data probably would not be successful because they could claim exemption on the basis of confidentiality of patient data. They also can't be asked for data they don't hold, which would include statistical analysis of the raw data that they haven't performed. So the art would seem to lie in asking for precisely the data which is there, and available, and not subject to any exemptions. If they claim exemption on the basis that they are going to publish, they can then be challenged on the basis that, in order to claim this exemption, they must then publish all the data requested, and not merely an analysis or commentary on it.

    I don't know the actual reference in the FOI legislation itself, but that would probably have to be looked up to remind the providers of their obligations. They will not release anything unless they are forced to do so.
  19. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    343
    UK
    Hi Mark
    I think if that what there is it is sufficiently vague for them to bat away requests with a decent lawyer. I don't see anybit that says they have to give a timetable, and in the case of recovery data they have said they will publish exactly that data.Personally, I think the MRC route looks more promising, or at least exploring if the 2005 rules apply to them. The MRC has given them £5m for a study that didn't come up with much, so they might feellike applying a little pressure, but that's probably optimism talking.
  20. Mark

    Mark Acting CEO

    Messages:
    4,527
    Likes:
    2,004
    Sofa, UK
    I agree insofar as the MRC may be a more effective route to apply pressure to get access to the data sooner rather than later. They have a clear stated policy, and the spirit of that policy is that they should release all data wherever possible and as soon as possible. It may be a route that gets access sooner, but it's not guaranteed to succeed.

    However the FOI route is more legally binding. Lots of information has come out under FOI that the holders really did not want to release. They can wriggle around, but in the end, they do have to comply. The terms as described in the JISC Questions are fairly clear that if they're asked to supply data (that is not exempt), they must do so, eventually, and publishing analyses based on that data is not enough. I don't know what time limits are built in, but there must be something in the legislation about that. Basically, once you ask them for data that is not exempt, they will then have to provide it. So unlike the MRC route, this is a legally mandatory route. There's no timetable mentioned in those questions, but there must be time limits in the legislation, they can't keep saying for ever "we will release it some day". I really think this must be achievable; it is not easy, it requires persistence and precisely the right questions, but I really do think it can be done.
    Enid likes this.

See more popular forum discussions.

Share This Page