LECTURE 4: EVALUATION OF HEALTH INTERVENTIONS
What is Evaluation?
• Evaluation has been defined as systematic investigation of the merit, worth, or
significance of an object.
• Systematic processes to determine the extent to which service needs and results have
been or are being achieved and analyse the reasons for any discrepancy.
• Attempts to measure service’s relevance, efficiency and effectiveness. It measures
whether and to what extent the programme’s inputs and services are improving the
quality of people’s lives.
Efficiency tells you that the input into the work is appropriate in terms of the output. This
could be input in terms of money, time, staff, equipment and so on. When you run a project and
are concerned about its replicability or about going to scale, then it is very important to get the
efficiency element right.
Effectiveness is a measure of the extent to which a development programmes or project
achieves the specific objectives it set. If, for example, we set out to improve the qualifications
of all the high school teachers in a particular area, did we succeed?
Impact tells you whether or not what you did made a difference to the problem situation you
were trying to address. In other words, was your strategy useful? Did ensuring that teachers
were better qualified improve the pass rate in the final year of school? Before you decide to get
bigger, or to replicate the project elsewhere, you need to be sure that what you are doing makes
sense in terms of the impact you want to achieve.
Evaluation is the only way to separate programs that promote health and prevent injury,
disease, or disability from those that do not; it is a driving force for planning effective
public health strategies, improving existing programs, and demonstrating the results of
resource investments. Evaluation also focuses attention on the common purpose of public
health programs and asks whether the magnitude of investment matches the tasks to be
accomplished.
,What is a health intervention?
This definition is broad so that almost any organized public health action can be seen as able to
benefit from program evaluation:
Categories of health interventions
•Direct service interventions (e.g., a program that offers free breakfasts to improve nutrition for
grade school children)
•Community mobilization efforts (e.g., an effort to organize a boycott of products to improve
the economic well-being of farm workers)
•Research initiatives (e.g., an effort to find out whether disparities in health outcomes based on
race can be reduced)
•Advocacy work (e.g., a campaign to influence the state legislature to pass legislation regarding
tobacco control)
•Training programs (e.g., a job training program to reduce unemployment in urban
neighborhoods)
Why evaluate Health programs?
Public health programs have as their ultimate goal preventing or controlling disease, injury,
disability, and death.
Some Reasons to Evaluate Public Health Programs:
, To monitor progress toward the program’s goals.
To determine whether program components are producing the desired progress on
outcomes.
To permit comparisons among groups, particularly among populations with
disproportionately high risk factors and adverse health outcomes.
To justify the need for further funding and support.
To find opportunities for continuous quality improvement.
To ensure that effective programs are maintained and resources are not wasted on
ineffective programs
There are many different ways of doing an evaluation. Some of the more common terms
you may have come across are:
_ Self-evaluation: This involves an organisation or project holding up a mirror to itself and
assessing how it is doing, as a way of learning and improving practice. It takes a very self-
reflective and honest organisation to do this effectively, but it can be an important learning
experience.
_ Participatory evaluation: This is a form of internal evaluation. The intention is to involve as
many people with a direct stake in the work as possible. This may mean project staff and
beneficiaries working together on the evaluation. If an outsider is called in, it is to act as a
facilitator of the process, not an evaluator.
_ Rapid Participatory Appraisal: Originally used in rural areas, the same methodology can, in
fact, be applied in most communities. This is a qualitative way of doing evaluations. It is semi-
structured and carried out by an interdisciplinary team over a short time. It is used as a starting
point for understanding a local situation and is a quick, cheap, useful way to gather information.
It involves the use of secondary data review, direct observation, semi-structured interviews, key
informants, group interviews, games, diagrams, maps and calendars. In an evaluation context, it
allows one to get valuable input from those who are supposed to be benefiting from the
development work. It is flexible and interactive.
_ External evaluation: This is an evaluation done by a carefully chosen outsider or outsider
team.
Interactive evaluation: This involves a very active interaction between an outside evaluator or
evaluation team and the organisation or project being evaluated. Sometimes an insider may be
included in the evaluation team.
ADVANTAGES AND DISADVANTAGES OF INTERNAL AND EXTERNAL
EVALUATIONS see page 9
Characteristics of a Good Evaluator
What is Evaluation?
• Evaluation has been defined as systematic investigation of the merit, worth, or
significance of an object.
• Systematic processes to determine the extent to which service needs and results have
been or are being achieved and analyse the reasons for any discrepancy.
• Attempts to measure service’s relevance, efficiency and effectiveness. It measures
whether and to what extent the programme’s inputs and services are improving the
quality of people’s lives.
Efficiency tells you that the input into the work is appropriate in terms of the output. This
could be input in terms of money, time, staff, equipment and so on. When you run a project and
are concerned about its replicability or about going to scale, then it is very important to get the
efficiency element right.
Effectiveness is a measure of the extent to which a development programmes or project
achieves the specific objectives it set. If, for example, we set out to improve the qualifications
of all the high school teachers in a particular area, did we succeed?
Impact tells you whether or not what you did made a difference to the problem situation you
were trying to address. In other words, was your strategy useful? Did ensuring that teachers
were better qualified improve the pass rate in the final year of school? Before you decide to get
bigger, or to replicate the project elsewhere, you need to be sure that what you are doing makes
sense in terms of the impact you want to achieve.
Evaluation is the only way to separate programs that promote health and prevent injury,
disease, or disability from those that do not; it is a driving force for planning effective
public health strategies, improving existing programs, and demonstrating the results of
resource investments. Evaluation also focuses attention on the common purpose of public
health programs and asks whether the magnitude of investment matches the tasks to be
accomplished.
,What is a health intervention?
This definition is broad so that almost any organized public health action can be seen as able to
benefit from program evaluation:
Categories of health interventions
•Direct service interventions (e.g., a program that offers free breakfasts to improve nutrition for
grade school children)
•Community mobilization efforts (e.g., an effort to organize a boycott of products to improve
the economic well-being of farm workers)
•Research initiatives (e.g., an effort to find out whether disparities in health outcomes based on
race can be reduced)
•Advocacy work (e.g., a campaign to influence the state legislature to pass legislation regarding
tobacco control)
•Training programs (e.g., a job training program to reduce unemployment in urban
neighborhoods)
Why evaluate Health programs?
Public health programs have as their ultimate goal preventing or controlling disease, injury,
disability, and death.
Some Reasons to Evaluate Public Health Programs:
, To monitor progress toward the program’s goals.
To determine whether program components are producing the desired progress on
outcomes.
To permit comparisons among groups, particularly among populations with
disproportionately high risk factors and adverse health outcomes.
To justify the need for further funding and support.
To find opportunities for continuous quality improvement.
To ensure that effective programs are maintained and resources are not wasted on
ineffective programs
There are many different ways of doing an evaluation. Some of the more common terms
you may have come across are:
_ Self-evaluation: This involves an organisation or project holding up a mirror to itself and
assessing how it is doing, as a way of learning and improving practice. It takes a very self-
reflective and honest organisation to do this effectively, but it can be an important learning
experience.
_ Participatory evaluation: This is a form of internal evaluation. The intention is to involve as
many people with a direct stake in the work as possible. This may mean project staff and
beneficiaries working together on the evaluation. If an outsider is called in, it is to act as a
facilitator of the process, not an evaluator.
_ Rapid Participatory Appraisal: Originally used in rural areas, the same methodology can, in
fact, be applied in most communities. This is a qualitative way of doing evaluations. It is semi-
structured and carried out by an interdisciplinary team over a short time. It is used as a starting
point for understanding a local situation and is a quick, cheap, useful way to gather information.
It involves the use of secondary data review, direct observation, semi-structured interviews, key
informants, group interviews, games, diagrams, maps and calendars. In an evaluation context, it
allows one to get valuable input from those who are supposed to be benefiting from the
development work. It is flexible and interactive.
_ External evaluation: This is an evaluation done by a carefully chosen outsider or outsider
team.
Interactive evaluation: This involves a very active interaction between an outside evaluator or
evaluation team and the organisation or project being evaluated. Sometimes an insider may be
included in the evaluation team.
ADVANTAGES AND DISADVANTAGES OF INTERNAL AND EXTERNAL
EVALUATIONS see page 9
Characteristics of a Good Evaluator
