Dr Corine Karema, Head of the Malaria and Other Parasitic Diseases Division (equivalent to National Malaria & NTDs Control Program) in the Rwanda Biomedical Center of the Rwanda Ministry of Health.
Dr Agnes Binagwaho, Minister of Health – Republic of Rwanda.
From June 17th to 19th 2013, Seattle hosted the Global Health Metrics and Evaluation (GHME) conference. The conference highlighted innovative methods, the latest debates in measurement, and the translation of data into effective policy to improve population health globally. A very well organized conference, no doubt, as it was co-hosted by institutions with extensive experience in both planning and running such event: the Institute for Health Metrics and Evaluation, The Lancet, the London School of Hygiene & Tropical Medicine, Harvard School of Public Health, and the University of Melbourne School of Population and Global Health.
Participants included researchers, academic leaders, students, policymakers, non-governmental organizations, foundations, country offices of health statistics, and national and multinational health organizations.
Dr Agnes Binagwaho, the Rwanda Minister of Health was invited to speak in the conference and share country level perspectives on health metrics, use of the Rwanda health information system, how they are being locally contextualised and how they inform effective policy for improved population health.
Unfortunately, the Rwanda Minister was unable to join the conference due to a last minute conflicting agenda and she asked Dr Paulin Basinga, Rwandan health expert, to share the Minister’s and collaborators’ results of an abstract that was published in the Lancet GHME conference proceedings.
The talk highlighted three discrepancies in Rwandan international estimates data that may have important policy implications:
First, as the Ministry was trying to evaluate national progress towards MDG 4, they found that there were serious differences in the numbers published by various groups working on Rwanda child mortality estimates. The Inter Agency Group on Child Mortality Estimation, using all of the latest data and most transparent methods, estimated that Rwanda reduced child mortality by 70% between 2000 and 2011, from 183 to 54 deaths per 1,000 live births.
However, other UN agencies had not updated their model inputs for several years, missing important new data that became available in 2009 and 2011 collected in the Rwanda DHS (2007-8 and 2010). They published these estimates in 2011 while their calculations’ estimates were based on data collected in 2004 and 2005!
Rwanda has been very proactive in using strategic information and evidence to guide any policy decision and is thus constantly checking updated available estimates, questioning some methodologies … after this discovery, it will be important to understand how many other countries have experienced the same thing. This kind of confusion is partially the result of limited data, but it is also connected to simple errors that could be fixed with a more collaborative approach to global health metrics and evaluation….
Second, the Ministry of Health in Rwanda observed that reports from international health agencies including WHO, UNICEF, and GAVI indicated a decline in vaccination coverage in Rwanda among infants under 12 months old. They knew that these figures were the result of modeling procedures seen as placeholders until Rwanda’s 2012 Census data is released. Due to the fact that Rwanda had introduced three new vaccines since 2009 and meticulously conducted assessments tied to these grants, they were quite confident that the coverage of vaccination among the infants under 12 months was above 95%. Yet, the international estimates showed coverage for Rwanda declining from the high 90s to approximately 80% — in just two years.
Attempting to understand the source of discrepancy, the team in Rwanda hypothesized that this was due to an artificially high denominator for the calculation: the number of children born each year in Rwanda. Upon close examination of the data, field evaluation, post vaccination assessment and methodologies, they observed that this was the case. The estimated birth cohorts used to calculate vaccination coverage did not take into account Rwanda’s increases in family planning uptake or reductions in total fertility rate over the past decade.
Just to be clear here, this discrepancy is not because of any lack of commitment to child survival by anyone in these agencies or in Rwanda; simply because of delayed updating of estimates, it was reported that Rwanda was failing to vaccinate 20% of its children — children who did not exist.
Using social media, a tweet went from the Minister’s account in June 2012, asking why there was a discrepancy between DPT3 coverage on the GAVI website and the DHS 2010 estimate.
The tweet led to several high-level phone calls, and shortly afterwards a technical delegation from WHO and UNICEF traveled to Kigali to revise the estimates in collaboration with the Ministry of Health’s EPI statistical team.
Routine dialogue and consultations over the inputs for estimates between countries and international organizations are crucial to ensure timeliness and accountability over key estimates.
Lastly, Rwanda has been one of the countries that warmly welcomed the publication of the 2010 Global Burden of disease study. The 2010 GBD and the continually updated GBD 2.0 offer an unprecedented opportunity to accelerate progress in tackling the right issues that contribute to important loss of DALYs. In a briefing on the country-level GBD results in Kigali in February 2013, the study was compared to the landmark achievement of landing a man on the moon for its potential to completely transform the way science and accountability in global health work.
As a result the GBD results are in active use in Rwanda, it has already contributed to a new program to avail more than 1 million new clean cookstoves across Rwanda in 2013, after GBD 2010 showed that household air pollution was the leading risk factor for DALYs.
But there remains great room for improvement. The most important area will be introducing real collaborations with country stakeholders in identifying the most relevant and up-to-date sources of data, as this will maximize accuracy and impact.
Independence and partnership are not mutually exclusive.
As one example, the GBD 2010 estimates for Rwanda show malaria as the leading cause of death for both men and women. IHME estimated that Rwanda had nearly 10,000 malaria deaths in 2010, but the Rwanda Health Management and Information System derived from data at the community and health facility level produced a WHO-validated estimate of only 670 deaths in 2010, down to 360 in 2011. The Ministry estimates are certainly not perfect, but they are certainly not off by more than an order of magnitude.
These discrepancies are due to the fact that regional averages were simply applied to all countries in East Africa, owing to a lack of easily accessible data for the researchers conducting the African malaria analysis: involving the Malaria control programs in countries would have prevented this error.
Rwanda has seen the potential of the GBD and the use of strategic information to support a new era of country ownership, transparency, and evidence based action. When these key ingredients are added together, they offer powerful ways to deliver on the promise of global health equity…. thus true collaboration between international organizations and country level policy makers is crucial to produce estimates that effectively inform country policies.