Levels of evidence
Grade 1 Evidence from systematic review of all relevant RCT
Grade 2 Evidence from 1 good RCT
Grade 3a Evidence from pseudo RCT
Grade 3b Evidence from comparative cohort studies, case control
studies or interrupted time series with control group
Level 3b Evidence from single arm studies, interrupted time series
without control group
Level 4 Evidence from case series
Phases of clinical trials
• Stage 1 (tens): evaluate safety, toleration, pharmacokinetics,
pharmacodynamics
• Stage 2 (hundreds): To determine efficacy and evaluate safety further
• Stage 3 (thousands): Compare intervention to other standard
interventions and monitor adverse effects and collect information to use
intervention safely
• Stage 4 (Population): Done after intervention is marketed to monitor
effectiveness of approved intervention in general population and other
adverse effects associated with widespread use for long periods of
time
Cluster RCT is a type of RCT in which groups of subjects as opposed to
individual subjects are randomised
Systematic Review
• What is a systematic review:
o Systematic and rigorous: minimise bias
o Answers a specific question (PICO)
o Protocol (Cochrane)
o Comprehensive search strategy documented
o Quality of studies appraised according to set criteria
o Summary of data
o Regularly updated (Cochrane)
• Other properties:
o Identification of records
o Screening and removal of duplicates
o Screening for eligibility
o Inclusion into meta analysis
Narrative Sytematic
Lack rigour Rigorous – minimise bias
•Broad in scope •Answers a specific question (PICO)
•Author can use literature to support •Protocol (Cochrane)
viewpoint •Comprehensive search strategy
•Limited critical appraisal documented
•Broad summary of ‘state of affairs’ •Quality of studies appraised
according to set criteria
•Summary of data
, •Regularly updated (Cochrane)
Potential biases
• Inclusion criteria: Too narrow
• Inadequate literature search
• Publication bias
• Inadequate assessment of study quality: number of tools available to
measure the quality of materials, including Cochrane
Statistical error
Type 1 Error • Test rejects true null hypothesis
• Rate of type 1 error is the given the value of α
• It usually equals the significance level of a test
Type 2 Error • Test fails to reject a false null hypothesis
• Rate of type 2 errors is given the value of β
• It is related to the power of the test
Goals:
• Minimise bias by discarding weak studies
• Maximise confidence about effectiveness of intervention
Literature search:
• PICO
o Patient or problem
o Intervention or indicator
o Comparison or control
o Outcome
• Focus: comprehensiveness
• Most efficient and accurate way: Carefully worded vocabulary
Funnel plots
Grade 1 Evidence from systematic review of all relevant RCT
Grade 2 Evidence from 1 good RCT
Grade 3a Evidence from pseudo RCT
Grade 3b Evidence from comparative cohort studies, case control
studies or interrupted time series with control group
Level 3b Evidence from single arm studies, interrupted time series
without control group
Level 4 Evidence from case series
Phases of clinical trials
• Stage 1 (tens): evaluate safety, toleration, pharmacokinetics,
pharmacodynamics
• Stage 2 (hundreds): To determine efficacy and evaluate safety further
• Stage 3 (thousands): Compare intervention to other standard
interventions and monitor adverse effects and collect information to use
intervention safely
• Stage 4 (Population): Done after intervention is marketed to monitor
effectiveness of approved intervention in general population and other
adverse effects associated with widespread use for long periods of
time
Cluster RCT is a type of RCT in which groups of subjects as opposed to
individual subjects are randomised
Systematic Review
• What is a systematic review:
o Systematic and rigorous: minimise bias
o Answers a specific question (PICO)
o Protocol (Cochrane)
o Comprehensive search strategy documented
o Quality of studies appraised according to set criteria
o Summary of data
o Regularly updated (Cochrane)
• Other properties:
o Identification of records
o Screening and removal of duplicates
o Screening for eligibility
o Inclusion into meta analysis
Narrative Sytematic
Lack rigour Rigorous – minimise bias
•Broad in scope •Answers a specific question (PICO)
•Author can use literature to support •Protocol (Cochrane)
viewpoint •Comprehensive search strategy
•Limited critical appraisal documented
•Broad summary of ‘state of affairs’ •Quality of studies appraised
according to set criteria
•Summary of data
, •Regularly updated (Cochrane)
Potential biases
• Inclusion criteria: Too narrow
• Inadequate literature search
• Publication bias
• Inadequate assessment of study quality: number of tools available to
measure the quality of materials, including Cochrane
Statistical error
Type 1 Error • Test rejects true null hypothesis
• Rate of type 1 error is the given the value of α
• It usually equals the significance level of a test
Type 2 Error • Test fails to reject a false null hypothesis
• Rate of type 2 errors is given the value of β
• It is related to the power of the test
Goals:
• Minimise bias by discarding weak studies
• Maximise confidence about effectiveness of intervention
Literature search:
• PICO
o Patient or problem
o Intervention or indicator
o Comparison or control
o Outcome
• Focus: comprehensiveness
• Most efficient and accurate way: Carefully worded vocabulary
Funnel plots
