Friday, 6 November 2015

Song for the Siren

There is another future waiting there for you
I saw it different, I must admit
I caught a glimpse, I'm going after it
They say people never change, but that's bullshit, they do
Yes I'm changing, can't stop it now
And even if I wanted I wouldn't know how
Another version of myself I think I found, at last
Yes I'm Changing by Tame Impala

Some 'follow-up' observations to my earlier 'Thoughts about Holes' post on the PACE follow-up study of Chronic fatigue Syndrome/ME by Sharpe et al 2015. 

To recap, after completing their final outcome assessment, some trial participants were offered an additional PACE therapy..."If they were still unwell, they wanted more treatment, and their PACE trial doctor agreed this was appropriate. The choice of treatment offered (APT, CBT, or GET) was made by the patient’s doctor, taking into account both the patient’s preference and their own opinion of which would be most beneficial.” White et al 2011
I have already commented on some of the issues about how these decisions were made, but here I focus on the Supplementary Material for the paper (see particularly Table C at the bottom of this post) and - what I believe to be some unsupported ...or unsupportable inferences made about the PACE findings recently.

Song to the Siren by Tim Buckely
(from the Monkees TV show)
Did I dream you dreamed about me?
Were you here when I was flotsom?
Now my foolish boat is leaning
Broken lovelorn on your rocks

I will start with three recent quotes making key claims about the success of the PACE follow-up findings and discuss the evidence for each claim.

1) First, in a Mental Elf blog this week, (Sir) Professor Simon Wessely rightly details and praises the benefits of randomised controlled trials (RCT), concluding that PACE matches-up quite well. But to extend Prof Wessely's nautical motif, I'm more interested in how 'HMS PACE' was seduced by the song of the Sirens, forced to abandon methodological rigor on the shallow rocky shores of bias and confound.

Prof Wessely states
"There was no deterioration from the one year gains in patients originally allocated to CBT and GET. Meanwhile those originally allocated to SMC and APT improved so that their outcomes were now similar. What isn’t clear is why. It may be because many had CBT and GET after the trial, but it may not. Whatever the explanation for the convergence , it does seem that CBT and GET accelerate improvement, as the accompanying commentary pointed out (Moylan, 2015)."
It seems to me that specific claims for "no deterioration from the one year gains in patients originally allocated to CBT and GET" might be balanced by the equally valid statement that we also saw "no deterioration from the one year gains in patients originally allocated to SMC and APT".  Almost one-third of the CBT and GET groups did, however, receive additional treatments as did the SMC and APT groups.  As I mentioned in my previous post, the mean group scores at follow-up are now a smorgasbord of PACE interventions, meaning that the group means lack... meaning!

At best, the PACE follow-up data might show that deterioration did not occur in the GET and CBT groups to any greater or lesser extent than it did in the SMC and APT groups. The original groupings effectively no longer exist at follow-up and we should certainly not flit between explanations sometimes based on initial randomised groupings and sometimes based on additional nonrandomised therapies.

In terms of deterioration, we know only one thing - one group were close to showing a significantly greater number of patients reporting 'negative change' during follow-up and contrary to the claim, this was the CBT group (see Table D in supplementary materials)

Yes I'm Changing by Tame Impala

2) Second, in the abstract of the PACE paper by Sharpe et al draw conclusions about long-term benefits of the original therapy groups:
"The beneficial effects of CBT and GET seen at 1 year were maintained at long-term follow-up a median of 2·5 years after randomisation. Outcomes with SMC alone or APT improved from the 1 year outcome and were similar to CBT and GET at long-term follow-up, but these data should be interpreted in the context of additional therapies having being given according to physician choice and patient preference after the 1 year trial final assessment." (my italics)

Again for the reasons just outlined, we cannot infer that CBT and GET maintained any benefits at follow-up anymore than we could argue that APT or even the control condition (SMC) maintained their own benefits. The smorgasbord data dish prevent any meaningful inference

3) Finally, in a Commentary that appeared in Lancet Psychiatry alongside the paper (mentioned by Prof Wessely above), Moylon and colleagues suggest hypotheses about the benefits of CBT and GET remain 'unproven' but that CBT and GET may accelerate improvement:
"The authors hypothesise that the improvement in the APT and SMC only groups might be attributed to the effects of post-trial CBT or GET, because more people from these groups accessed these therapies during follow-up. However, improvement was observed in these groups irrespective of whether these treatments were received, and thus this hypothesis remains unproven. ....Overall, our interpretation of these results is that structured CBT and GET seems to accelerate improvement of self-rated symptoms of chronic fatigue syndrome compared with SMC or SMC augmented with APT..."

Round and Round by Ariel Pink's Haunted Graffiti
It's always the same, as always
Sad and tongue tied
It's got a memory and refrain
I'm afraid, you're afraid
And we die and we live and we're born again
Turn me inside out
What can I say...
Merry go 'round
We go up and around we go
Moylon et al rightly point out that improvement occurred "irrespective of whether these treatments were received, and thus this hypothesis remains unproven" . Although not apparent from the main paper, the supplementary material throws light on this issue, however, Moylon et al are only half right!

We can see in Table C of the supplementary material (see below) that those in the CBT group, APT and SMC showed significant improvements even when no additional therapies were provided - so, Moylon are correct on that score.  By contrast, the same cannot be said of the GET group. At follow-up, GET shows no significant benefit on measures of fatigue (CFQ) or of physical function (SF-36PF) ...whether they received additional adequate therapy, partial therapy or indeed, no further therapy!

This is even more interesting when we consider that Table C reveals data for 20 GET patients (16%) who had subsequently received 'adequate CBT' - and it clearly produced no significant benefits on their fatigue or their physical function scores. So, what are we to conclude? That CBT is ineffective? CBT is ineffective following GET? That these patients are 'therapy resistant'? Therapy resistant because they received GET?

Whatever the explanation, GET is the only group to show no improvement during follow-up. Even with no additional therapy, both the SMC controls and the APT group improved, as indeed did CBT. The failure of GET patients to respond to adequate additional CBT therapy is curious, not consistent with the claims made and does not look 'promising' for either GET or CBT.

The inferences described above do appear to be holed below the water line.

1) CBT does not appear to accelerate improvement least in people who have previously received GET
2) People who received GET show no continuing improvement post-therapy
and 3) CBT may heighten the incidence of 'negative change' .




  1. Thank you again for covering this topic. Ever since Professor Wessely compared the PACE trial to a cruiseliner leaving Southampton for a rocky yet prosperous vogage, patients have been comparing it to the Titanic, with journalist David Tuller and Professor James Coyne as icebergs. I like your siren analogy, though – something certainly pulled PACE off its course.

    I hope you won’t mind me mentioning that in the last nine days, over 8,000 people have signed a petition calling for the retraction of meaningless claims of improvement and recovery made in PACE on the basis of analyses that replaced the per-protocol analyses.

    In the new analyses, patients were considered to have recovered their physical function if they scored at similar levels to people with Class II congestive heart failure.

    The brief, clear background pages to the petition make grim but fascinating reading.

    Emeritus Professor Jonathan Edwards of University College London has called PACE “valueless”.

    1. I forgot to ask people to join in and sign the petition! That's ME-related brainfog for you. The target is 10,000 signatures - less than 2,000 to go now!

  2. Excellent article ... it's great to see the inaccurate statements and speculations getting the scrutiny that they deserve!

    Thank you!

  3. I feel that there may be another point which is being missed. We don't know when additional treatments happened. For CBT and GET they basically tell patients to ignore symptoms believe they can do stuff and they will get better. The measurements taken basically ask about what a patient can do and fatigue symptoms. So I would have thought that CBT and GET may well change the questionnaire responses even without changing actual symptoms and function.

    The point about timing comes in terms of how long such effects would last and if someone was currently in say CBT would there response be further biased?

  4. Brilliant ! And thanks very much for looking into the whole PACE issue.

  5. Thanks for your thoughts on this flawed study.

  6. Thanks for your assessment of this flawed study.

  7. You say "1) CBT does not appear to accelerate improvement least in people who have previously received GET"

    As I understand it those undergoing CBT in the main phase were also receiving newsletters encouraging them that they were undergoing proven treatment. Presumably those who had received GET in the main phase and later received CBT were receiving the latter treatment without the ongoing encouragement of these newsletters. As CBT is about encouraging subjects to change their attitudes to illness and that newsletters can also have a similar function, this makes a difference to the total stimulus to a change of attitude.

  8. This comment has been removed by a blog administrator.