The evaluation can be exhaustive (analysis of each activity and strategies) or focus on certain activities and/or strategies.
It examines different aspects: effectiveness, accessibility, responsiveness, safety and quality, and resource mobilisation.
As soon as the intervention begins, define the objectives and indicators, make sure that the tools needed to collect the data are available and that the personnel are trained in using them.
Some indicators are collected routinely throughout the outbreak. Others are collected during field visits at a sample of health care facilities or vaccination sites, according to a specific methodology and with specific tools established at the start of the intervention. The information is compiled and analysed when the epidemic is over.
A final intervention evaluation report is written and presented to the epidemic management committee. It is a critical analysis of the operations conducted and offers recommendations for improving the next response. It is based on the evaluation tables below.
7.5.1 Evaluation of surveillance
Indicator | Data needed |
Source/ collection tools |
Method | Comments |
Operational efficacy | ||||
---|---|---|---|---|
Percentage of facilities that have the case definition |
|
|
Visit to a sample of health care facilities |
Pay particular attention to peripheral facilities
Expected result: 100% |
Percentage of facilities that send weekly surveillance forms |
|
|
Exhaustive, for the duration of the outbreak |
Expected result: 100% |
Time to transmit weekly surveillance forms |
|
|
Exhaustive, for the duration of the outbreak |
Expected result: |
Laboratory surveillance | ||||
% positive samples |
|
|
Exhaustive, for the duration of the outbreak, in sentinel districts |
|
Time to laboratory confirmation (time from identification of first cases to laboratory confirmation) |
|
|
|
Expected result: |
AEFI surveillance | ||||
AEFI surveillance exists |
|
|
Interview | |
Incidence of serious AEFIs |
|
|
|
|
Breakdown of serious AEFIs by cause |
|
|
|
7.5.2 Evaluation of patient care
Indicator | Data needed |
Source/ collection tools |
Method | Comments |
Effectiveness | ||||
---|---|---|---|---|
Reported case fatality |
|
|
Analysis of measles |
Easily measured if the
Expected result: < 5% |
Overall CFR and specific CFR rate by age and by facility |
|
|
|
Eliminates bias due to an unreliable surveillance system
Expected result:
|
Accessibility | ||||
Percentage of functional facilities that are supplied with treatments during the outbreak |
|
|
Detailed analysis of donation forms and the list of health care facilities |
Verify that all health care facilities are functional
Expected result: 100% |
Percentage of facilities that have the treatment protocol |
|
|
Visit to a sample of health care facilities |
Protocol in national language
Expected result: 100% |
Percentage of facilities |
|
|
or
|
Systematic analysis at the district level
Expected result: |
Responsiveness | ||||
Time to supply specific treatments |
|
|
Detailed analysis of the:
|
Pay particular attention to peripheral facilities
Expected result: 1 to 2 weeks |
Time from alert in the health zone to supply of specific treatments to district facilities |
|
|
Detailed analysis of the:
|
Pay particular attention to
Expected result: 1 week |
Security/Quality | ||||
Percentage of cases treated according to the recommended protocol |
|
|
On a sample of facilities, analysis of:
|
Pay particular attention to peripheral facilities
Expected result: 100% |
Injection safety |
|
|
Visit to a sample of facilities and observation |
Expected result: 100% |
Cost | ||||
Cost per patient treated |
|
|
These costs include drugs, supplies, transport and personnel |
Requires preparation with the accounting staff |
7.5.3 Evaluation of vaccination
Indicator | Data needed |
Source/ collection tools |
Method | Comments |
Effectiveness | ||||
---|---|---|---|---|
Vaccine effectiveness |
|
|
Several methods:
At one or several selected locations |
Done by an epidemiologist (see Section 7.6)
|
Number of cases prevented by vaccination Preventive fraction |
|
|
|
Done by an epidemiologist |
Accessibility | ||||
Vaccination coverage by |
|
|
|
Expected result:
|
Percentage of sites that did not experience a vaccine or ADS shortage |
|
|
or
|
At the end of the campaign:
Expected result: 100% |
Responsiveness | ||||
Time from the epidemic alert to the start and end of the vaccination campaign |
|
|
|
Calculated at the end of the campaign
Expected result: |
Number of people vaccinated per day and per team |
|
|
or
|
Analyse urban and rural areas separately
Expected result:
|
Security/Quality | ||||
Vaccine utilisation rate |
|
|
Exhaustive for the entire length of the campaign |
Expected result: ≥ 85% (i.e., wastage ≤ 15%) |
Ratio of ADS used/ number of safety boxes used |
|
|
or
|
The ratio should not be greater than the maximum capacity of the safety boxes used
Expected result:
|
Percentage of personnel suffering needlestick injury during the campaign (AEB) |
|
|
or
|
If questionnaire used, verify that recommended AEB procedure was followed |
Percentage of refrigerators with up-to-date temperature monitoring sheet |
|
|
or
|
Expected result: 100% |
Percentage of vaccination sites with a proper waste collection and disposal system |
|
|
|
During the campaign
Expected result: 100% |
Cost | ||||
Overall cost of the campaign |
|
|
Analysis of expenditures: vaccines, injection supplies, transport, personnel, cold chain, logistics, etc.) |
|
Cost to vaccinate one person |
|
|
Analysis of expenditures |
Requires preparation with the accounting staff |
Cost per case and death prevented |
|
|
Analysis of expenditures | After the campaign |
Resources | ||||
Ratio of ADS used/ number of vaccines administered |
|
|
|
Expected result: the ratio should not be less than 1 |
Percentage of teams with an appropriate number of people |
|
|
or
|
Expected result: 100% |
7.5.4 Evaluation of social mobilisation
Indicator | Data needed |
Source/ collection tools |
Method | Comments |
Efficiency/Operational efficacy | ||||
---|---|---|---|---|
Percentage of reasons for non-vaccination related to lack of information |
|
Vaccination coverage survey with study of reasons for non-vaccination |
Vaccination coverage survey |
Done at the end of the campaign
Expected result: < 10% |