The evaluation can be exhaustive (analysis of each activity and strategies) or focus on certain activities and/or strategies.
It examines different aspects: effectiveness, accessibility, responsiveness, safety and quality, and resource mobilisation.
As soon as the intervention begins, define the objectives and indicators, make sure that the tools needed to collect the data are available and that the personnel are trained in using them.
Some indicators are collected routinely throughout the outbreak. Others are collected during field visits at a sample of health care facilities or vaccination sites, according to a specific methodology and with specific tools established at the start of the intervention. The information is compiled and analysed when the epidemic is over.
A final intervention evaluation report is written and presented to the epidemic management committee. It is a critical analysis of the operations conducted and offers recommendations for improving the next response. It is based on the evaluation tables below.
7.5.1 Evaluation of surveillance
Indicator | Data needed |
Source/ collection tools |
Method | Comments |
Operational efficacy | ||||
---|---|---|---|---|
Percentage of facilities that have the case definition |
– Number of facilities where the case definition is available |
– List of heath care facilities |
Visit to a sample of health care facilities |
Pay particular attention to peripheral facilities Expected result: 100% |
Percentage of facilities that send weekly surveillance forms |
– Number of facilities that send the surveillance form each week |
– Weekly tracking form for reception of surveillance data |
Exhaustive, for the duration of the outbreak |
Expected result: 100% |
Time to transmit weekly surveillance forms |
– Date weekly surveillance forms sent |
– List of heath care facilities |
Exhaustive, for the duration of the outbreak |
Expected result: |
Laboratory surveillance | ||||
% positive samples |
– Number of samples taken by type of test requested |
– Laboratory test register |
Exhaustive, for the duration of the outbreak, in sentinel districts |
|
Time to laboratory confirmation (time from identification of first cases to laboratory confirmation) |
– Date and location of first cases |
– Weekly surveillance form |
– Analysis of weekly surveillance forms or registers from health care facilities |
Expected result: |
AEFI surveillance | ||||
AEFI surveillance exists | – Existing surveillance system |
– District chief medical officer, person in charge of surveillance |
Interview | |
Incidence of serious AEFIs |
– Number of AEFIs by age group and location for the period |
– Individual AEFI reporting form |
– Exhaustive |
|
Breakdown of serious AEFIs by cause |
– Total number of serious AEFIs by cause |
– Individual AEFI reporting form |
– Exhaustive |
7.5.2 Evaluation of patient care
Indicator | Data needed |
Source/ collection tools |
Method | Comments |
Effectiveness | ||||
---|---|---|---|---|
Reported case fatality |
– Number of cases and deaths by administrative unit (region, district, etc.) |
– Measles surveillance Excel file |
Analysis of measles |
Easily measured if the Expected result: < 5% |
Overall CFR and specific CFR rate by age and by facility |
– Number of cases and deaths recorded by age and by facility for the epidemic period |
– Register of measles cases |
– Calculated for each hospital |
Eliminates bias due to an unreliable surveillance system Expected result: |
Accessibility | ||||
Percentage of functional facilities that are supplied with treatments during the outbreak |
– Number of facilities supplied by administrative unit |
– Donation forms |
Detailed analysis of donation forms and the list of health care facilities |
Verify that all health care facilities are functional Expected result: 100% |
Percentage of facilities that have the treatment protocol |
– Number of facilities where the protocol is available |
– Protocol present in the facility |
Visit to a sample of health care facilities |
Protocol in national language Expected result: 100% |
Percentage of facilities |
– Inventory shortage noted at the district level: date and duration. |
– Stock cards |
– Detailed analysis of stock cards, donation forms and measles treatment availability Excel file |
Systematic analysis at the district level Expected result: |
Responsiveness | ||||
Time to supply specific treatments |
– Date first cases reported |
– Measles surveillance Excel file |
Detailed analysis of the: |
Pay particular attention to peripheral facilities Expected result: |
Time from alert in the health zone to supply of specific treatments to district facilities |
– List of health care facilities supplied and date |
– Measles surveillance Excel file |
Detailed analysis of the: |
Pay particular attention to Expected result: |
Security/Quality | ||||
Percentage of cases treated according to the recommended protocol |
– Number of cases |
– Recommended protocol |
On a sample of facilities, analysis of: |
Pay particular attention to peripheral facilities Expected result: 100% |
Injection safety |
– Number of facilities using safety boxes |
– Evaluation grid |
Visit to a sample of facilities and observation |
Expected result: 100% |
Cost | ||||
Cost per patient treated |
– Total cost of the curative component of intervention |
– Invoices |
These costs include drugs, supplies, transport and personnel |
Requires preparation with the accounting staff |
7.5.3 Evaluation of vaccination
Indicator | Data needed |
Source/ collection tools |
Method | Comments |
Effectiveness | ||||
---|---|---|---|---|
Vaccine effectiveness |
– Case definition |
– Register of measles cases or line listing |
Several methods: |
Done by an epidemiologist (see Section 7.6)
|
Number of cases prevented by vaccination Preventive fraction |
– Demographic data |
– Measles surveillance Excel file |
– On a sample of locations |
Done by an epidemiologist |
Accessibility | ||||
Vaccination coverage by |
– Number of doses |
– Measles vaccination summary Excel file |
– Analysis of collected data |
Expected result: |
Percentage of sites that did not experience a vaccine or ADS shortage |
– Date and duration of inventory shortages at the district level |
– Stock cards |
– Analysis of documents |
At the end of the campaign: Expected result: 100% |
Responsiveness | ||||
Time from the epidemic alert to the start and end of the vaccination campaign |
– Date of the alert |
– Measles surveillance Excel file |
– Exhaustive if possible |
Calculated at the end of the campaign Expected result: |
Number of people vaccinated per day and per team |
– Duration of the campaign: date by location (separate urban and rural areas) |
– Tally sheet |
– Exhaustive if possible |
Analyse urban and rural areas separately Expected result: |
Security/Quality | ||||
Vaccine utilisation rate |
– Number of doses injected |
– Measles vaccination summary Excel file |
Exhaustive for the entire length of the campaign |
Expected result ≥ 85% |
Ratio of ADS used/ number of safety boxes used |
– Number of ADS used |
– Measles vaccination summary Excel file |
– Exhaustive |
The ratio should not be greater than the maximum capacity of the safety boxes used Expected result: |
Percentage of personnel suffering needlestick injury during the campaign (AEB) |
– Number of people suffering a needlestick injury during the campaign |
– AEB reporting form |
– Exhaustive analysis (if AEB reporting in place) |
If questionnaire used, verify that recommended AEB procedure was followed |
Percentage of refrigerators with up-to-date temperature monitoring sheet |
– Number of refrigerators used for the campaign |
– List of refrigerators used for vaccine storage |
– On a random sample of locations |
– At a minimum, monitor the district cold chain Expected result: 100% |
Percentage of vaccination sites with a proper waste collection and disposal system |
– Total number of sites |
– Observation grid |
– Sample of sites |
During the campaign Expected result: 100% |
Cost | ||||
Overall cost of the campaign |
– Total expenditures | – Accounting of expenditures |
Analysis of expenditures: vaccines, injection supplies, transport, personnel, cold chain, logistics, etc.) |
|
Cost to vaccinate one person |
– Total amount of vaccination activities (urban/rural areas) |
– Measles vaccination summary Excel file |
Analysis of expenditures |
Requires preparation with the accounting staff |
Cost per case and death prevented |
– Total amount of vaccination activities |
– Measles surveillance Excel file |
Analysis of expenditures | After the campaign |
Resources | ||||
Ratio of ADS used/ number of vaccines administered |
– Number of ADS used |
– Measles vaccination summary Excel file |
– Overall in the district |
Expected result: the ratio should not be less than 1 |
Percentage of teams with an appropriate number of people |
– Number of people per team and appropriate qualifications for task |
– Observation grid |
– On a random sample of sites |
– Standard team composition defined at the start of the campaign (urban and rural) Expected result: 100% |
7.5.4 Evaluation of social mobilisation
Indicator | Data needed |
Source/ collection tools |
Method | Comments |
Efficiency/Operational efficacy | ||||
---|---|---|---|---|
Percentage of reasons for non-vaccination related to lack of information |
– Total number of unvaccinated persons |
Vaccination coverage survey with study of reasons for non-vaccination |
Vaccination coverage survey |
Done at the end of the campaign Expected result: < 10% |