How NOT to Conduct a Randomized Clinical Trial

How NOT to Conduct a Randomized Clinical Trial


well good afternoon everybody and
welcome to biostatistics in action tips for clinical research a lecture series
this series of talks is organized by CTSA in collaboration with department of
biostatistics. Our special guest today is Dr. Bruce Levin professor
and past chair in the department of biostatistics Dr. Levin has extensive
expertise in designing and consulting for clinical trials and today he will
present a very interesting study and point out how NOT to conduct a
randomized clinical trial without further ado Dr. Levin thank you thanks
very much for coming on this hot and humid day. Cody does have printed
handouts of my talk so you don’t need to take notes if you prefer not to and although I am inclined typically to give
technical biostatistical talks I promise you today there won’t be a single
formula and this is a non-technical talk nevertheless it’s going to raise some
subtle issues about the conduct of a randomized controlled trial I want to
say that I’m going to be using one trial the PACE trial which will describe in a
moment as the vehicle because as it turns out it is a great teaching example
for how a well-designed well conceived randomized control trial can go off the
rails and cause havoc and controversy and raise serious ethical concerns so
although it’s about one trial there were many learning points that I hope to
point out during my talk I want to give some heartfelt acknowledgments first and
foremost to David teller is an investigative journalist and until
recently ran a joint program in public health and journalism at the University
of California Berkeley without his courageous investigated with journalism
I don’t think we would have the knowledge and understanding quite the
same way that we do today much more than an acknowledgement you may discover that
I’ve actually plagiarized much of David tellers writing but I wrote to him and
told him about this acknowledgement and confession and hopefully he’ll
understand, as well Carolyn Wilshire for her profound psychological insights
I am a co-author am honored to be a co-author of a paper she and others
wrote to reanalyze the original clinical end points for the PACE trial to see
what happened and therein hangs a tale to Vince Racaniello, our own professor of
biology for putting up the many blogs that David and others have written about
this subject if you don’t know Virology Blog please check it out and to many
many ME/CFS patients especially Alem Matthees and Julie Rehmeyer for
conversations and profound insights as well so for background in February of 2011
United Kingdom researchers published a paper in The Lancet entitled comparison
of adaptive pacing therapy cognitive behavior therapy graded exercise therapy
and specialist medical care for chronic fatigue syndrome the acronym of that
trial being PACE a randomized trial the authors are P.D. White, M.C. Sharpe, T. Chalder,
and others listed in the citation below one comes with a presumption
perhaps that a main article major article published in the lancet we have
great credibility and should be given serious attention and so it was let me
digress for a moment throughout today’s talk there’ll be a complete
schizophrenia between British spelling of words like behavior with a u or
randomized with an S and the American spelling with out that you with a Z or
Zed as my Canadian friends say as opposed to an S um that’s because I’ve
quoted liberally shall we say from the original writing so sometimes I’m
quoting a British version sometimes my own American version The Lancet article
reported that two rehabilitative treatments cognitive behavioral therapy
CBT for short and graded exercise therapy
GET or get we’re safe and effective treatments for patients with a
particular syndrome known as in the old days chronic fatigue syndrome today more
appropriately called myalgic encephalomyelitis or Emmys or mu CFS for
short the effects were statistically
significant and seemed quite substantial when you read the paper the paper is
here if anyone would like to see it after the
talk again let me offer a disclaimer I am NOT a clinician I’m a biostatistician
I have a PhD not an MD and so if I appear to be knowledgeable about the
syndrome it’s only because I’ve learned these words you know I play one on TV
and I call on the more clinically knowledgeable people in today’s audience
to correct me if I make any errors as a result of this Lancet paper the well in
fact let me circulate the paper as we speak the PACE trial received a lot of
attention international attention and had widespread impact on research actual
treatments prescribed for patients and attitudes toward the illness both in the
medical community and the patient community and Republican large for
example what got a great deal of attention was at the time of the
publication of the 2011 Lancet paper there was a press conference held by the
investigators to promote the Lancet paper and one of the lead investigators
Trudy Chalder stated and we quote here twice as many people on graded exercise
therapy and cognitive behavioral therapy got back to normal she used the word
normal compared to those in the other two study arms the other two study arms
was a modality called adaptive pacing therapy and the control arm was
specialized medical care what we might think of the standardized medical care
or standard of care but yes referred explicitly to specialized medical care
if I call it the control group that’s what it is its function and purpose was
to serve as control in today’s talk I’m going to focus just on CBT and get not
so much on adaptive pacing for the simple reason that in the original trial
adaptive pacing therapy looked almost identical to the control arm and so
there’s very little there there but I just wanted to remind you that this was
a forearm randomized control trial furthermore an accompanying commentary
in The Lancet claimed similarly that these back to normal participants had
met a strict criterion for recovery when you could read those comments at the
time the trial so this is important stuff and
you have to pay attention to it I’d like to try to clarify what the underlying
hypothesis the investigators had which justified in their minds the use of
cognitive behavioral therapy and the other modality graded exercise therapy
which was that any patients Harbor unhelpful convictions about having an
ongoing organic disease and that the perpetuation of their devastating
symptoms is the result of deconditioning therefore it would be plausible that CBT
or GET could help in other words if patients are afraid of having relapses
when they exercise or exert themselves in any manner they’re going to be afraid
to do so and their muscles become deconditioned and it involves a vicious
cycle so if only we can intervene cognitively to change their behavior to
alleviate their fear maybe we can get them past the psychological impediments
for rehabilitating this theory somewhat unkindly something I call it’s all in
your head stands in stark contrast to a recent Institute of Medicine now the US
National Academy of Medicine review in 2015 the IOM reported that ME/CFS is a
complex multi system illness characterized by neurological
immunological or autonomic and energy metabolism dysfunctions
terms of diagnosis the Cardinal symptom that these patients exhibit is systemic
intolerance to exertion if patients exceed their available energy resources
they can suffer serious and prolonged relapses those of you familiar with this
disease know that it’s devastating sometimes patients can’t get out of bed
we’ve seen films documentary films where patients had to excruciating ly crawl up
the stairs just to get into bed and then could not get out for weeks on end
unfortunately the etiologic mechanism underlying of the disease is still
unknown there are hypotheses that it’s all in
the mind there are other hypotheses that this is some unknown sequela of the
viral infection there’s some evidence for that because onset of such symptoms
often follow viral illness a number of years ago our own in looking here at
Columbia was involved in an NIH funded study to confirm or refute the
hypothesis that the specific viral agent might be XMRV, Xenotropic Murine
Leukemia related virus, it’s a type of virus that has symptoms of leukemia in a
mouse model but that turned out not to be the case somewhat unfortunately
because everyone was hoping this might be the causal agent but in our study I
was involved in designing with 150 cases of ME/CFS and 150 controls in the
retrospective design we look for the prevalence of XMRV in cases relative to
controls well the answer was there was no virus found either in the controls
nor of the cases so this is certainly not an explanation
a more believable explanation is that the assay for the XMRV had to be so
exquisitely sensitive that it was picking up mouse molecules in the saliva
when the mice went into the warehouse for the laboratory assay reagents and
started nibbling through the burlap sacks believe it or not left a few Mouse
molecules and that looks like XMRV so this is certainly a subtle disease not
so subtle in its manifestations but certainly of unknown etiology currently
other hypotheses are that its exposure to mold toxins Julie Rehmeyer for
example there’s written a book in which she purports to have been alleviated of
her symptoms when she went to a stark mold-free
environment in the desert in Arizona you can imagine though that if this PACE
trial were not armor-plated foolproof and credible patients may have some
concern about the underlying theory of the rehabilitative modalities in a
nutshell patients have said and I believe them what do you mean this is
all in my head let me tell you this is not in my head I cannot move and there’s
something wrong with me it’s not a psychosomatic illness so I’m here to
discuss certain peculiarities that were brought to light thanks to the vocal
participants of the PACE trial in the United Kingdom about some troubling
peculiarities this is perhaps one of the most
outrageous peculiarity and one in my entire career I had never seen before in
terms of bio statistical concepts patients baseline scores for the two
primary outcomes one being physical function ability and we’ll talk about
how that’s measured and fatigue and we’ll talk about how that’s measured
could qualify them simultaneously as sufficiently disabled to qualify as
eligible for the trial after all you have to have eligibility criteria for a
trial of this syndrome for patients but as well already recovered the day you’re
enrolled in the trial as eligible for treatment you can be recovered that’s
weird this happened because the criteria for
recovery were changed months after the data collection was completed sometime
between when the statistical analysis plan which is required for a formal
phase 3 randomized control trial sometime between when the analysis plan
was finalized and the publication in 2011 came out something happened the the
endpoint for recovery was changed now the investigators claimed that this was
done before they looked at the data and with the approval of the ethics
Oversight Committee but as good skeptical biostatisticians we have to
ask ourselves a few questions here first of all the PACE trial was not masked or
blinded regarding treatment group the revised end points were subjective
self-reported outcomes they were not the original Lee Priest
to find objective measures as stated in the trial protocol it was not clear to
what extent the Oversight Committee was independent of the investigators it was
a three-person Oversight Committee and what their relationships were is unclear
and troubling to me as a biostatistician the statistical analysis plan wasn’t
even published until years later so okay you’re antennas go up what’s going on
here let me tell you something about the end points here so they were there was
one primary end point in the original protocol called improvement and there
was a major pre-specified secondary endpoint of recovery improvement was a
composite binary outcome comprising several criteria that we’ll get into in
a moment and some time a year after the data were collection was locked and
before the publication in this unblinded trial the primary endpoint was changed
into two continuous measures. One of the measures was the Chalder Fatigue
Questionnaire. Trudy Chalder was one of the co-authors of the
PACE publication she created this questionnaire to measure fatigue and it
was changed from a binary end point to a continuous measure it’s a quantitative
scale the other component was the physical function sub scale of the sf-36
common quality of life measure you’re probably familiar with here why why were these endpoints changed we
can have a conversation about that later but let me continue the description here
recovery was the primary secondary I’m sorry the main secondary endpoint and it
too originally was pre specified as a composite measure based on four criteria
one involving the same Chalder Fatigue Questionnaire the same SF-36
physical function subscale plus criteria related to a clinical global impression
and also various definitions of cases as you can imagine what constitutes a
positive diagnosis of ME/CFS has evolved over our familiarity with the syndrome
and it is not a trivial or obvious type of diagnosis again the criteria for
those components changed mysteriously between the end of the trial essentially
and the publication I don’t want to spend too much getting lost in the
details of the measures but I do want to run quickly through it so that you’ll
see and know what I’m talking about here so for the primary endpoint one
component was a minimum score of 75 on the sf-36
100 point physical function subscale or increase in the physical function score
toward the better end of 50% or more compared to baseline and some other
components which I’ll get to in the next slide but this particular criterion was
changed into at least an eight point increase in the hundred point sf-36 scan
so now it’s getting a little bit hard to calibrate how do you translate a minimum
score of 75 or a 50% increase into what does an 8 point increase me that’s the
first component the next component originally was of the 11 fatigue related
items in the Chalder Fatigue Questionnaire three or fewer which the
patient would self rate as worse or much worse than prior to the illness onset or
the total number of items rated were so much worse dropped by at least 50%
so that was the original criteria it was changed to at least a two-point decrease
in the 33 point Chalder Fatigue Questionnaire again moving to a a more
continuous flavor in addition the next component you had to qualify as having a
minimum score of 85 on the sf-36 oh I’m sorry that ended the the changed
criteria there were two criteria for the primary endpoint of improvement with
respect to recovery which remember got a lot of publicity
there were several components one is a minimum score of 85 on the sf-36
physical function scale that changed to a minimum score of 60 so they made it a
lot easier to recover in this particular component originally you also had to
have three or fewer rated as work worse or much worse on the fatigue
questionnaire that now changed to a maximum score of 18 on the 33-point CFQ next in the clinical mobile impression
scale the CGI originally you have to self rate as very much better now your
overall health had to be either much better or very much better
and finally originally the case definition was met if the patient no
longer fulfilled for recovery purposes the Oxford case definition of CFS the
CDC criteria and the London Emmy criteria those there were three groups
Oxford CDC here in the US and London group and if the patient met any of
those they would not be considered recovered originally they had to not be
considered a case anymore on all three in order to be considered recovered by
the time the publication came out now the patient didn’t meet the standard
Oxford case definition or on the Chalder Fatigue Questionnaire they
rated less than six of the eleven fatigue items as being worse or their
sf-36 physical function score was greater than 65 so again it’s hard to
know just by reading these changes what exactly we should expect from this but
you can see that there were major substantial changes to this definition
so as a result of these changes fully thirteen percent of patients were
already recovered and then that got exaggerated into within the normal range
at the start of the study for self-reported physical function in other
words congratulations you are eligible with your consent you are enrolled in
the trial Oh congratulations you’re now recovered that’s weird right and this is
what they teller brought to my attention
originally he said have you ever seen anything like this and I said have to be
honest no I’ve never seen anything like that it seems rather amateurish to me
but it gets worse the investigators did not publish the findings for the
original end points as specified in the protocol even as a sensitivity analysis
you would think that if there is an originally specified statistical
analysis plan and for whatever cogent reasons you had to make a change in the
definition of the end points you owe it to the public and the scientific
community as well as the patients to report what the original end points
might have been and then have a conversation about if and whether there
was any differences in the conclusions they didn’t do that in fact when
patients for other peculiarities ins started to complain and began to demand
release of the data the investigator is dug in their heels and actually accused
the patient’s requests as vexatious vexatious is a very interesting word you
know it has a colloquial sense if somebody’s being vexatious you know they
are annoying right but it also is a term of art in the law if a lawsuit is
classified as being vexatious it means that it’s a frivolous lawsuit whose sole
purpose is to harass the respondent so when they called these requests
vexatious they were essentially threatening their own participants with
legal recourse as we already mentioned the PACE trial
claims of successful improvement and recovery were based solely on subjective
outcomes notwithstanding the fact that the original protocol did have objective
measures there were four objective measures a a walking tests the
six-minute walk test how far can you walk in a six-minute interval a fitness
test related to oxygen consumption it’s the vo2 max by a step test data
from governmental databases on days loss from work six months after the primary
endpoint was measured and the fourth objective measure was the receipt of
financial benefits for disability those original objective measures fail to
provide any evidence to support claims of efficacy for any of those treatments
so again you’re you’re scratching your head and saying so wait what’s going on
here there was one objective measure I should correct myself and that was the
six-minute walk test which found that get but not CBT participants walked
reliably meaning significantly further or father than control participants at
the time of the primary endpoint which was measured 52 weeks after
randomization so if CBT and get were both found to be significantly better
for improvement in recovery how come only one of those manifested in the
six-minute walk test but not the other but for those knowledgeable about this
particular test the quantitative difference was that the get group walked
on average 67 meters farther than they did on baseline and the controls walked
about 30 meters father and baseline so there’s already what you
might call a placebo effect or an improvement in fact even in the controls
and so you want to wonder you know is that forty seven meters
significant meaning clinically significant to put it in context a
sample of class two chronic heart failure patients with similar baseline
walking distances as those in the PACE trial increase their distance by an
average of 141 meters after only three weeks of a much gentler greater than
exercise program so to put it in context okay this was a significant finding but
you might say well you know does this qualify to getting back to normal but it
gets worse I hope you you’re gathering for my negative effect that changing
primary endpoints and secondary endpoints in a complex way without a
great deal of cogent compelling justification is something you should
not do in a randomized control trial there may be statistical arguments for
example if it turns out upon interim analyses that the power of the test is
nowhere near as high as was assumed during the planning stages then a very
careful very rigorous data safety monitoring board examination is entitled
to approve a major change like this but again there are much more profound
difficulties for doing this in the context of the unblinded and
self-reported outcome type of measurement and we’ll get to
that later in the talk here’s another question David teller put
to me in the middle of the study the PACE team published a newsletter for
participants that included glowing testimonials from earlier trial subjects
about how much the therapy and treatments helped them the newsletter
also included an article informing participants that the two interventions
pioneered by the investigators oh wait a second so the two main investigators
were pioneers in the development of these interventions a the two
interventions CBT and yet had been recommended as treatments by a UK
government committee quote based on the best available evidence now I’m not
exactly sure what evidence they’re talking about one of the ethical
principles of randomized control trials as I’m sure you understand is that there
must be clinical equipoise in the community about with a palpable lack of
consensus about which is the best treatment everybody is familiar with
that definition of clinical equipoise less well known but equally important is
a second part of the principle of clinical equipoise which is that
anything that you do that would threaten the ability of the trial to perturb the
state of clinical equipoise and affect ongoing medical practice is unethical
this is the reason why phase 3 randomized control trials are supposed
to be adequately powered in other words have a large enough sample size because
you need that large sample size in order to compel or have compelling evidence
to the clinical community that one treatment is superior to the other and
then get doctors to move to that evidence base or on the contrary to have
a large enough sample size to conclude that no statistically significant
difference actually means there’s a very trivial and unimportant clinical
difference so if you don’t have a large enough sample size you’re violating that
ethical principle and it can be said legitimately that the trial is unethical
I would argue similarly changes such as the ones we’ve been discussing today
which threaten the validity of the trial and which therefore inhibit the ability
to perturb the state of clinical equipoise is similarly unethical but you
know we don’t have to get as subtle as that that newsletter didn’t mention that
a keep ace investigator was also serving on the very same government committee
that endorsed the PACE therapies not cool even without that conflict of
interest issue I would never consider publishing a newsletter in the middle of
the trial the recruitment had not even been complete yet and then never mind
the follow-up that announced that two of the therapies were now approved by a
government committee as being endorsed as therapy for this illness why because
you’re going to ruin your ability to recruit patients what patient would
agree to be randomized in their trial where a government authoritative body
has just approved two of the therapies but not the others it threatened to
destroy the very first leg of that clinical equipoise principle and because
of that I was quoted in the media as calling the conduct of this trial the
height of amateurism and I stand by that opinion it gets
worse the PACE protocol contained an explicit commitment to tell prospective
participants about conflicts of interest the investigators had long-standing
financial and consulting ties with disability insurance companies having
advised them for years that CBT and get could get claimants off benefits and
back to work okay this happens but guess what
wasn’t stated in the informed consent form nobody nobody was told about this
potential conflict of interest when the patient’s learned about this they were
furious they were furious and rightly so all right so plenty of
funny things going on here so what happened two years ago one of the
patients whom I acknowledged, Alem Matthees, is an Australian patient, sued the home
institution of the PACE investigators Queen Mary University of London you know
freedom of information action, a UK FOIA request to get the original
lease specified end points released initially the PACE investigators refused
nope not going to do it why not they were very concerned with protecting the
confidentiality of the identity of the patients you know if we give out the
data you know some unscrupulous vexatious person might be identify the
data and find them out a tribunal was assembled and the
tribunal decided to side with the patient, Alem Matthees. Queen Mary
University of London appealed nope not going to do it a second tribunal was
formed and the second tribunal again came down this time with a court order
you shall release those data and so they did except they didn’t release all the
data they withheld certain demographic variables which they claimed could still
be is for deidentification purposes but they did release the unstratified data so
in the original trial it was a stratified randomization technique that
was being used and of course there’s a principle of trial analysis that says if
you randomized within strata you should analyze the data similarly within strata
those of us involved in the reanalysis of the original endpoints could not
replicate the stratified analysis because we weren’t given the variables
that would tell us what stratum the patient belonged to I don’t think that
made much of a difference because in a randomized study because you’re
randomizing within strata when you ignore the stratification you don’t get
a lot of bias typically but there is the potential for small differences
nevertheless so some of the data were released and analyses of those data have
confirmed that the extensive outcome switching allowed the investigators to
report dramatically better findings than the null or the minimum
results obtained with the originally pre- specified measures
specifically on the original protocol specified primary outcome measure
overall improvement there was a significant effect of treatment group
now I’m not denying the existence of the statistical significance of that group
difference however the groups receiving CBT or get did not significantly
outperform the control group after correcting for the number of comparisons
as was pre specified in the statistical analysis plan let me try to disentangle
a little bit what’s going on here so remember when the investigators switched
the primary outcome they created two continuous measures so in the PACE
publication that’s circulating you’ll see that there was such and such a
quantitative mean point differential between the two groups but the authors
did also publish the binary components which had also been revised by the
changes in those criteria that I showed you earlier
so in all reanalysis the best we could do was to compare the original criteria
that went to the composite binary with the new criteria that were published and
that’s what we’re talking about here in the original sap the statistical
analysis plan the investigators said they would quote significance
after a bun furrow knee adjustment for six comparisons because they had six
endpoints and that’s what they said they would do in the publication somehow
mysteriously that six went to five we don’t know why bond for only six change
to bond for only five but it did and I’ll show you some numbers
moment but before we get too hung up over whether something’s significant or
whether it survives a Bonferroni adjustment for multiple comparisons the
elephant in the room here is potential bias and I want to finish the rest of my
talk with an examination of potential bias here but just to tell you give you
a flavor of what happened in the published report for the primary
endpoint of overall improvement the CBT group 59% of them qualified compared to
45 in the control group and that difference was slightly larger than a
p-value of 0.01 so if you multiply by 6 or even 5 you aren’t allowed to claim
statistical significance but in the trial protocol with the original
definitions there was still a difference of about 10 percentage points the CBT
and get get about 20% of the patients with the original definitions as
compared to 10% and again P values of 0.01 5 or 0.01 technically not
statistically significant what take-home message do you bring from this situation
technically not significant but it looks it looks significant there’s an
important element of any scientific endeavor which is a-priori-ism if you
say you’re going to do something and you get funded to do it and approved by all
kinds of review committees then you want to do it you’re not allowed to wiggle
out of it and say oh but look the p-value is 0.01 never mind the fact that
we said the p-value adjusted would be 0.06 or whatever
it’s not good science not good statistics recovery is even more
shocking yeah so the p-value here is the unadjusted p-value if you just compare
20% for CVT with 10% for standard that you know doing a two-by-two table that
gives you a p-value of 0.01 5 but you can’t claim that as statistically
significant until you multiply by 6 according to the original protocol so it
should have been stated as not significant P equals 0.065 or
whatever the product is the rates of recovery as the main secondary endpoint
we’re consistently low and not significantly different across the
treatment groups so here you see in the published report they got 22 percent of
patients having recovered great compared to only 7 percent in the control group
but when you use the original more stringent criteria which patients tell
me by the way that original criterion was much closer to what a patient or a
clinician would consider as having been recovered or partially recovered the
rates were small 7 percent 4 percent versus 3 percent and certainly not
significant so what happened here was by this redefinition it allowed the
investigators to massively exaggerate the success of the trial results I’m not
saying that CBT does not help patients I’m saying that the characterization of
the results the exaggeration of the results and
spin that these investigators have applied to the results are not really
justifying on other secondary measures significant effects were almost entirely
confined to self-report measures and everything disappeared after a two-year
long term follow-up period the primary endpoint was measured at one year 52
weeks but there was a long-term follow-up in a subset of patients and
the effects had gone away and there’s a whole controversy as well which I can’t
get into for lack of time about how come the effects disappeared after two years so Carolyn Wilshire and Tom Kinland and
Alem Matthees and myself as co-authors published in the British BMC psychology that these reanalysis findings raise
serious concerns about the robustness of the claims made about the efficacy of
CBT the modest treatment effects we wrote obtained on self-report measures
in the PACE trial do not exceed what could be reasonably accounted for by
participant reporting biases let me quickly go over those biases because
these are perhaps the most important things to note heavy reliance on self
reports from participants who are aware of their treatment allocation raises
concerns now of course in behavioral intervention trials full blending is
often not possible like in the PACE trial but in that situation it is the
researchers responsibility it behooves the researchers to consider the possible
effects of the lack of blinding on outcomes especially subjective
self-report outcomes and to ensure that such factors would not rise
to the level where it might account for the apparent findings of benefit in the
trial in a trial that’s not blinded self-reported outcomes it is well known
can produce highly inflated estimates of treatment related benefits and I would
argue this trial is a good example of that given that there’s a known
discrepancy in the effects of blinding on subjective versus objective measures
it appears unlikely that the improvement and recovery effects set to say reflect
genuine health benefits we feel a much more plausible explanation is that their
expectation related artifacts that reflect the operation of attentional
biases that favor the reporting of events consistent with one’s
expectations possibly recall or confirmation bias that enhance
recollection for events as consistent with the patient’s expectations
now that sounds complicated mouthful what does that mean and you know full
disclosure the PACE investigators just discount that alternative explanation
they’ve argued that expectancy effects alone cannot account for the positive
self reported improvements because by randomization patients expectations were
the same across the groups but what they fail to point out is that these two
interventions CBT and get the participants were primed during the
treatment period to expect improvement the manual given to the patients for CBT
proclaim CBT to be a powerful and safe treatment which has been shown to be
effective in this illness the get participants patient manuals said it’s
one of the most effective therapies currently known and both interventions
emphasize that faithful adherence to the program could lead to a full recovery
but guess what you know you think you raise patients
expectations yeah I think so and guess what no such
statements were given to the other two treatment groups is if that isn’t
grounds for sufficient bias and this is my last substantive point here the very
treatment itself of CPT and to a lesser extent get was that you should stop
focusing on your symptoms let’s get off this notion that your symptoms are
stressful and and debilitating to you you know if we can only get you properly
psychologically conditioned you’ll feel a whole lot better
so stop reporting all these bad symptoms you know don’t focus on it no such
instruction were given to the other group the control room
and so guess what when your primary endpoint ask patients to self-report
their symptoms what do you think is going to happen
it’s outrageous actually to wrap up here are some consequences the story is far
from over in fact David teller emailed me just before today’s talk to say that
The Times of London and the British Medical Journal have both picked up
stories news stories about the topics we’re discussing here and in particular
his open letter to The Lancet Leonard number three to the editor of The Lancet
but even before that up to date the AHRQ downgraded its recommendations for CBT
and get the downgraded grading occurred because the agency actually removed the
PACE trial from its meta-analysis the PACE investigators claim that get is
safe remember graded exercise therapy is you slowly
raised the level of exercise you don’t overdo it you let the patient acclimate
and that’s safe but AHRQ beg to differ they said that
yet was associated with more adverse events the US CDC abandoned its
recommendations that patients be treated with these two therapies a couple of
months later the UK National Institute of Health and Care excellence announced
that it would undertake a major reanalysis I don’t know what outcomes
they found yet and I’m interested to see the Dutch health council recommended get
should not be used in the Netherlands as a treatment and just this past March a
leading group of American clinicians who specialized in the illness unanimously
agreed that the two PACE treatments are inappropriate and possibly harmful for
patients with the illness and should therefore not be prescribed so these are
pretty serious consequences and I think it shows that the ability of this trial
to perturb the state of clinical equipoise failed because of
inappropriate conduct of the trial and as I said in my opinion that’s unethical
so I want to thank you very much if you want to read everything that I said
today go to Vince Racaniello Virology Blog there’s the URL and you
can read about it in David’s own words in particular the third open letter to
The Lancet there’s a whole other story about why the editor John Horton of The
Lancet has refused to even entertain critical publications to have a
scientific debate but that’s a whole other story so thank you very much
[Applause]

You May Also Like

About the Author: Oren Garnes

1 Comment

  1. Good to see a Stats POV on this, this seems a terrible case of data dredging given the lack of any appropriate blinding. Originally stats were given the groupings (ABCD), as stated in the limited SAP, but data collected on treatment times etc would probably have unblinded these. Why wasn’t this data handled separately? Preliminary results mean the blind was broken early, and as ABCD groupings known to Stats then all subsequent patients unblinded. So much potential for bias

Leave a Reply

Your email address will not be published. Required fields are marked *