Editorial
Large simple trials in psychiatry: providing reliable answers
to important clinical questions
JOHN R. GEDDES
THE EMERGENCE OF A GOLD STANDARD Institute for Mental Health in the United States and the
Medical Research Council in the UK. These trials were
The need for reliable evidence for medical interven- often concerned with major questions regarding classes
tions has long been recognised and the history of attempts of treatments, rather than specific compounds. Notable
to produce this evidence is thoroughly documented in among these early trials were RCTs investigating the
the James Lind library (http://www.jameslindlibrary efficacy of antipsychotic medication (National Institute
.org/). The status of randomised controlled trials as the of Mental Health Psychopharmacology Service Center
most reliable method for evaluating the effects of treat- Collaborative Study Group, 1964) and antidepressant
ments is now well established. Although the basic prin- drugs and electroconvulsive therapy (Medical Research
ciples of the randomised trial have a long developmen- Council, 1965). During the next decades until the early
tal history, it is often considered that the first modern 1980s, a number of other trials were conducted investi-
medical randomised controlled trial came was the UK gating key issues about the newly introduced drug
Medical Research Council trial of streptomycin in the treatments in psychiatry, notably long-term treatment
treatment of tuberculosis (Medical Research Council, with antipsychotics (Leff & Wing, 1971), continuation
1948). In the years following this trial, many ran- treatment with antidepressants (Mindham et al., 1972),
domised trials were conducted on a number of key treatment of acute mania with lithium and chlorpro-
questions in a number of disease areas including psy- mazine and prevention of relapse with lithium in both
chiatry and mental health. Many of these were funded bipolar and unipolar disorders (Prien et al., 1973a, b).
by national public bodies, particularly National From the 1980's the situation changed considerably
and in the area of pharmaceutical treatments, public
funding declined and the vast majority of clinical trials
of drug treatments in psychiatry are now sponsored and
conducted by the pharmaceutical industry. This change
had a number of effects. Firstly, the number of trials
has increased dramatically. Secondly, trials were
Address for correspondence: Professor J.R. Geddes, Department of increasingly conducted by industry on single patented
Psychiatry, University of Oxford. Warneford Hospital. Oxford 0X3 7JX
(United Kingdom). compounds and are designed to meet the requirements
Fax: +44 (0)18-65 79.3101 of the regulatory bodies such as the Food and Drug
E-mail: Administration (FDA) in the US and the Medicines and
Declaration of Interest: The BALANCE Trials are funded mainly
by the Stanley Medical Research Institute. JG has also received addi- Healthcare Products Regulatory Agency (MHRA) in
tional research support from Sanofi-Aventis and Glaxo-Smith-Kline for the UK. Although the criteria for the regulatory bodies
the independently conducted randomised trials described. I have is that the drugs should be safe and effective, the design
received research grants from the UK Medical Research Council. UK
Department of Health, UK National Health Service and the Stanley of trials that are conducted for regulatory purposes
Medical Research Institute. I have also accepted honorariums for work- often involves clinical compromises in order not to
shops and lectures from Sanofi-Aventis. Glaxo-Smith-Kline and Lilly delay the introduction of drugs onto the market. The
which are used to fund independent research. I am a member of the
National Institute of Clinical Excellence Health Technology Appraisal majority of such trials are short-term, highly controlled,
Committee.
Epidemiologia e Psichiatria Sociale, 14, 3, 2005
122
Downloaded from https://www.cambridge.org/core. IP address: 184.73.56.143, on 09 Dec 2021 at 15:28:48, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1121189X00006357
, Large simple trials in psychiatry: providing reliable answers to important clinical questions
highly monitored typically designed to separate out to be less effective - or more harmful - than existing
between new drug and placebo as efficiently as possi- treatments (Geddes, 2003; Geddes et al, in press).
ble. This approach to clinical trial design has been Secondly, the recommendations may effectively and
described as explanatory (Schwartz & Lellouch, 1967) inappropriately end the condition of equipoise, making
in which the aim is to determine if the treatment can any further trials extremely difficult to conduct. Three
work. Patients in these trials are characteristically high- UK examples show some of the problems that can arise
ly selected, follow up is short and outcomes are essen- when short-term commercial trials are used to produce
tially sensitive proxies of clinically relevant outcomes. policy:
For example, the prototypical trial of a new antidepres-
sant drug targeted at gaining regulatory approval is Atypical antipsychotics
placebo controlled, with or without an active compara- In the UK, the systematic review (http://www.nice.
tor (which is usually a currently marketed reference org.uk/pdf/AssessmentReport-v2-221001.pdf) undertak-
antidepressant drug), of six weeks duration, and en for the National Institute of Clinical Excellence
excludes patients with psychiatric or physical co-mor- appraisal of the new atypical antipsychotics reported:
bidity or suicidal ideation. The primary outcome mea- Evidence for the effectiveness of the new 'atypical'
sure is the change on a rating scale of depressive symp- antipsychotics compared to older drugs is, in general,
toms (usually the Hamilton depression rating scale) at ofpoor quality, based on short term trials and difficult
the end of six weeks. Loss to follow up is usually sub- to generalise to the whole population of people with
stantial (around 30%) because of the stringent protocol, schizophrenia. Evidence for the effectiveness of the
and intention-to-treat analysis is accomplished by using new 'atypical' antipsychotics compared to each other
the last observation carried forward. is still limited. Evidence for the cost-effectiveness of
the new 'atypical' antipsychotics in the UK compared
to each other and to older drugs is also limited. All
THE LIMITATIONS OF REGULATORY TRIALS conclusions are therefore based on limited evidence
FOR CLINICAL DECISION-MAKING and should be treated with caution.
The NICE Appraisal Committee considered this report
Because the design of these trials is based on a negoti- but essentially ignored these reservations and recom-
ation between regulators and industry, these trials often mended that atypical drugs should be a first line therapy
have substantial limitations for answering clinical ques- option for patents with schizophrenia. These recommen-
tions in the real world and, hence, limited clinical credi- dations then became official NHS policy - and made it
bility. Nonetheless, when they are the only randomised very difficult to conduct the trials that had been also iden-
evidence available, they will often be used to answer clin- tified as necessary within the same report.
ical questions for which they were not designed. For
example, evidence-based clinical practice guidelines and Anticholinesterase inhibitors
health technology assessments (which characteristically The anticholinesterase inhibitors (including rivastig-
seek to provide recommendations about the optimal aver- mine, donepezil and galantamine) were introduced in the
age treatment policy for unselected patient in several 1990's to treat Alzheimer's Disease. As in the case of the
clinical subgroups) are often ill served by the results of atypicals the UK National Institute of Clinical Excellence
these commercial trials. While the limitations of the considered the evidence review (http://www.nice.org.uk
available trials are often emphasised in the systematic /pdf/Alzheimer_hta.pdf)- Although the review found that
reviews that are undertaken to inform the policy state- the randomised evidence was of reasonable quality and
ment, the guideline developers understandably wish to confirmed improvement in cognitive impairment, the tri-
base their recommendations on the best available evi- als were of unknown applicability and the cost-effective-
dence and so they will often choose to use the existing ness of the drugs was very uncertain. The review also
randomised evidence rather than no evidence at all. The noted that a large simple trial - AD2000 (commissioned
results of the available evidence often need to be extrap- and funded by the National Health Service) - was in
olated - to an unknown degree - when they are applied to progress. Rather than defer a conclusion until AD2000
the target clinical question. Two main problems emerge reported, the NICE Appraisals Committee recommended
when policymakers go beyond the available evidence. that anticholinesterase inhibitors should be made avail-
First of all, such recommendations may be premature. able to patients on the NHS. This recommendation orded
The drugs may not be cost-effective and may even turn equipoise and recruitment to AD2000 became very diffi-
Epidemiologia e Psichiatria Sociale, 14, 3, 2005
123
Downloaded from https://www.cambridge.org/core. IP address: 184.73.56.143, on 09 Dec 2021 at 15:28:48, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1121189X00006357
Large simple trials in psychiatry: providing reliable answers
to important clinical questions
JOHN R. GEDDES
THE EMERGENCE OF A GOLD STANDARD Institute for Mental Health in the United States and the
Medical Research Council in the UK. These trials were
The need for reliable evidence for medical interven- often concerned with major questions regarding classes
tions has long been recognised and the history of attempts of treatments, rather than specific compounds. Notable
to produce this evidence is thoroughly documented in among these early trials were RCTs investigating the
the James Lind library (http://www.jameslindlibrary efficacy of antipsychotic medication (National Institute
.org/). The status of randomised controlled trials as the of Mental Health Psychopharmacology Service Center
most reliable method for evaluating the effects of treat- Collaborative Study Group, 1964) and antidepressant
ments is now well established. Although the basic prin- drugs and electroconvulsive therapy (Medical Research
ciples of the randomised trial have a long developmen- Council, 1965). During the next decades until the early
tal history, it is often considered that the first modern 1980s, a number of other trials were conducted investi-
medical randomised controlled trial came was the UK gating key issues about the newly introduced drug
Medical Research Council trial of streptomycin in the treatments in psychiatry, notably long-term treatment
treatment of tuberculosis (Medical Research Council, with antipsychotics (Leff & Wing, 1971), continuation
1948). In the years following this trial, many ran- treatment with antidepressants (Mindham et al., 1972),
domised trials were conducted on a number of key treatment of acute mania with lithium and chlorpro-
questions in a number of disease areas including psy- mazine and prevention of relapse with lithium in both
chiatry and mental health. Many of these were funded bipolar and unipolar disorders (Prien et al., 1973a, b).
by national public bodies, particularly National From the 1980's the situation changed considerably
and in the area of pharmaceutical treatments, public
funding declined and the vast majority of clinical trials
of drug treatments in psychiatry are now sponsored and
conducted by the pharmaceutical industry. This change
had a number of effects. Firstly, the number of trials
has increased dramatically. Secondly, trials were
Address for correspondence: Professor J.R. Geddes, Department of increasingly conducted by industry on single patented
Psychiatry, University of Oxford. Warneford Hospital. Oxford 0X3 7JX
(United Kingdom). compounds and are designed to meet the requirements
Fax: +44 (0)18-65 79.3101 of the regulatory bodies such as the Food and Drug
E-mail: Administration (FDA) in the US and the Medicines and
Declaration of Interest: The BALANCE Trials are funded mainly
by the Stanley Medical Research Institute. JG has also received addi- Healthcare Products Regulatory Agency (MHRA) in
tional research support from Sanofi-Aventis and Glaxo-Smith-Kline for the UK. Although the criteria for the regulatory bodies
the independently conducted randomised trials described. I have is that the drugs should be safe and effective, the design
received research grants from the UK Medical Research Council. UK
Department of Health, UK National Health Service and the Stanley of trials that are conducted for regulatory purposes
Medical Research Institute. I have also accepted honorariums for work- often involves clinical compromises in order not to
shops and lectures from Sanofi-Aventis. Glaxo-Smith-Kline and Lilly delay the introduction of drugs onto the market. The
which are used to fund independent research. I am a member of the
National Institute of Clinical Excellence Health Technology Appraisal majority of such trials are short-term, highly controlled,
Committee.
Epidemiologia e Psichiatria Sociale, 14, 3, 2005
122
Downloaded from https://www.cambridge.org/core. IP address: 184.73.56.143, on 09 Dec 2021 at 15:28:48, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1121189X00006357
, Large simple trials in psychiatry: providing reliable answers to important clinical questions
highly monitored typically designed to separate out to be less effective - or more harmful - than existing
between new drug and placebo as efficiently as possi- treatments (Geddes, 2003; Geddes et al, in press).
ble. This approach to clinical trial design has been Secondly, the recommendations may effectively and
described as explanatory (Schwartz & Lellouch, 1967) inappropriately end the condition of equipoise, making
in which the aim is to determine if the treatment can any further trials extremely difficult to conduct. Three
work. Patients in these trials are characteristically high- UK examples show some of the problems that can arise
ly selected, follow up is short and outcomes are essen- when short-term commercial trials are used to produce
tially sensitive proxies of clinically relevant outcomes. policy:
For example, the prototypical trial of a new antidepres-
sant drug targeted at gaining regulatory approval is Atypical antipsychotics
placebo controlled, with or without an active compara- In the UK, the systematic review (http://www.nice.
tor (which is usually a currently marketed reference org.uk/pdf/AssessmentReport-v2-221001.pdf) undertak-
antidepressant drug), of six weeks duration, and en for the National Institute of Clinical Excellence
excludes patients with psychiatric or physical co-mor- appraisal of the new atypical antipsychotics reported:
bidity or suicidal ideation. The primary outcome mea- Evidence for the effectiveness of the new 'atypical'
sure is the change on a rating scale of depressive symp- antipsychotics compared to older drugs is, in general,
toms (usually the Hamilton depression rating scale) at ofpoor quality, based on short term trials and difficult
the end of six weeks. Loss to follow up is usually sub- to generalise to the whole population of people with
stantial (around 30%) because of the stringent protocol, schizophrenia. Evidence for the effectiveness of the
and intention-to-treat analysis is accomplished by using new 'atypical' antipsychotics compared to each other
the last observation carried forward. is still limited. Evidence for the cost-effectiveness of
the new 'atypical' antipsychotics in the UK compared
to each other and to older drugs is also limited. All
THE LIMITATIONS OF REGULATORY TRIALS conclusions are therefore based on limited evidence
FOR CLINICAL DECISION-MAKING and should be treated with caution.
The NICE Appraisal Committee considered this report
Because the design of these trials is based on a negoti- but essentially ignored these reservations and recom-
ation between regulators and industry, these trials often mended that atypical drugs should be a first line therapy
have substantial limitations for answering clinical ques- option for patents with schizophrenia. These recommen-
tions in the real world and, hence, limited clinical credi- dations then became official NHS policy - and made it
bility. Nonetheless, when they are the only randomised very difficult to conduct the trials that had been also iden-
evidence available, they will often be used to answer clin- tified as necessary within the same report.
ical questions for which they were not designed. For
example, evidence-based clinical practice guidelines and Anticholinesterase inhibitors
health technology assessments (which characteristically The anticholinesterase inhibitors (including rivastig-
seek to provide recommendations about the optimal aver- mine, donepezil and galantamine) were introduced in the
age treatment policy for unselected patient in several 1990's to treat Alzheimer's Disease. As in the case of the
clinical subgroups) are often ill served by the results of atypicals the UK National Institute of Clinical Excellence
these commercial trials. While the limitations of the considered the evidence review (http://www.nice.org.uk
available trials are often emphasised in the systematic /pdf/Alzheimer_hta.pdf)- Although the review found that
reviews that are undertaken to inform the policy state- the randomised evidence was of reasonable quality and
ment, the guideline developers understandably wish to confirmed improvement in cognitive impairment, the tri-
base their recommendations on the best available evi- als were of unknown applicability and the cost-effective-
dence and so they will often choose to use the existing ness of the drugs was very uncertain. The review also
randomised evidence rather than no evidence at all. The noted that a large simple trial - AD2000 (commissioned
results of the available evidence often need to be extrap- and funded by the National Health Service) - was in
olated - to an unknown degree - when they are applied to progress. Rather than defer a conclusion until AD2000
the target clinical question. Two main problems emerge reported, the NICE Appraisals Committee recommended
when policymakers go beyond the available evidence. that anticholinesterase inhibitors should be made avail-
First of all, such recommendations may be premature. able to patients on the NHS. This recommendation orded
The drugs may not be cost-effective and may even turn equipoise and recruitment to AD2000 became very diffi-
Epidemiologia e Psichiatria Sociale, 14, 3, 2005
123
Downloaded from https://www.cambridge.org/core. IP address: 184.73.56.143, on 09 Dec 2021 at 15:28:48, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1121189X00006357
