Concept and objectives

Child health programmes in low-income countries are ultimately justified in term of their impact on child survival and how they may contribute to reaching the Millennium Development Goals. The international news and the scientific policy debate are therefore also full of assessments indicating that particular programmes saved a certain number of children. For example, the vaccination programme is said to have saved over 20 million lives in the last two decades and the vitamin A programme over a quarter of a million lives each year (http://www.child-survival.org/childsurvival/whatiscs.php). Such assessments are mostly based on measurements of performance indicators for a particular programme, e.g. vaccination coverage, and assumptions about efficacy of the intervention and assumptions about the burden of the particular health problem (disease or deficiency). The assumptions about efficacy of intervention are mostly based on small-scale pilot studies of the immediate target condition; for example, a vaccine is evaluated for its clinical protection against the targeted disease. 

This approach is clearly insufficient. First, there is little follow-up of how the programmes are implemented in real life and how this impacts on efficacy. Second, there is no consideration of unintended positive or negative health side-effects of the common health interventions. Third, possible sex-differences in response to the interventions are not assessed. Fourth, the possibility that health interventions may interact because they all affect the immune system is not considered.

Observational studies and randomised trials conducted by the Bandim Health Project in several African countries have consistently shown that when it comes to assessment of real life effects of an intervention, extrapolation of results from small-scale pilot studies is not reliable. First, vaccine and micronutrients are often given out of the recommended schedule and this can have marked consequences for the effect on mortality. Second, vaccines and micronutrients have beneficial or negative non-specific effects; i.e. effects which are not explained by prevention of the targeted infections or deficiencies. Third, these effects are frequently sex-differential. Fourth, interventions may interact producing stronger beneficial or negative non-specific effects; for example, vitamin A given together with DTP may increase mortality for girls. 

Hence, there is a need to assess the real life impact of the existing child health intervention programmes in the places where they are being used, taking into account the variability in programme implementation, that vaccines and micronutrients can have non-specific sex-differential effects on mortality, and that interventions may interact. Only real-life estimates of effects can lead to valid assessment of cost-effectiveness and thereby to evidence-based sound policy making. A prerequisite for testing the real life impact and cost effectiveness of our child health interventions is to have individual-based data on health intervention uptake as well as on health status. Most low-income countries do not collect individual data on health interventions and morbidity and mortality. However, many countries have Health and Demographic Surveillance Systems (HDSS) sites. The INDEPTH Network is a network of such sites in Africa and Asia, and has 37 members. These sites have typically been initiated as demographic surveillance sites, collecting data on births, deaths, and migration, but in recent years have integrated collection of health data into the routine data collection. The research from the Bandim Health Project HDSS has proved that HDSS sites can be extremely valuable tools for monitoring and testing the real life effects of health interventions.

In the present project we intend to take advantage of the HDSS sites in Africa. Based on a few modifications of the current data collection at these sites, including the routine collection of information on all interventions in childhood, such as vaccines, micronutrient supplementation, de-worming, bed-net distribution and bed-net impregnation, given at the health centres or in campaigns, these HDSS sites can provide the platform for 1) assessing the real life effect and cost-effectiveness of child health intervention programs in observational studies; 2) testing modifications of the current health intervention programs in randomised controlled trials (RCT), and 3) testing new interventions and their potential interactions with existing interventions in RCTs. This methodology will be implemented at three sites, at the Bandim Health Project in urban and rural Guinea-Bissau, in Nouna, Burkina Faso, and in Navrongo, Ghana. In the future this method can easily be adapted by other HDSS sites in Africa and Asia. 

Already implemented health interventions can generally only be evaluated in observational studies, because once a WHO-recommended intervention has been established in collaboration with international donors, it would be unethical to conduct a randomised trial of its “real life effect” since it would imply withholding an intervention considered beneficial from some children. Hence, when it comes to testing the real life effect of the current child health program we rely on observational studies for assessing impact and cost-effectiveness of health programmes. Such studies are prone to bias and the hypotheses need to be formulated in ways that minimise the risk of bias. A recent paper has formulated four testable hypotheses regarding the potential non-specific effects of the currently used diphtheria-tetanus-pertussis (DTP) and measles vaccines and their potential sex-differential effects (33). Using the HDSSs as a platform we will test these hypotheses. The comparison of children who get an intervention with children who did not get the intervention is likely to be affected by different forms of bias and we will therefore emphasise the assessment of the determinants of programme adherence - from socio-economic factors to health related determinants like nutritional status or previous history of infections - and control for these factors in the analyses. 

In some situations it is ethically justified to test modifications of the current programmes in RCTs and obtain unbiased estimates of the effect of certain aspects of a health programme. This may be useful because it evaluates the impact of possible modifications in an unbiased way but also because it may cast light on interactions between different interventions and test the observations made in the observational studies. The HDSS platform is ideal for conducting such RCTs and we intend to implement this possibility in the present project. Specifically, we want to test a recent finding from a randomised trial in Guinea-Bissau: providing early measles vaccine at 4.5 and 9 months of age compared with the recommended measles vaccine at 9 months of age reduced overall mortality between 4.5 and 36 months of age by an astonishing 49% (31). 

The HDSS sites already collect data on mortality, and mortality would be the main outcome in the present evaluation. Fortunately, mortality has declined in Africa in the last decade and it may become increasingly difficult to measure the overall impact on mortality of existing and new interventions. We therefore also aim to identify the best relevant comparable outcome parameters which correlate with child mortality/survival and which can be used to assess the overall health impact of interventions in future assessment.