This reflection is part of a blog series for the Methodological lessons and learning in longitudinal research project. Over coming months, Young Lives will share reflections on its methodology captured in interviews, blogs, videos, podcasts and reports designed to support building a community of practice in longitudinal research in low- and middle income countries, and open a dialogue. We welcome your feedback and your involvement.
Since Young Lives was founded in 2001, donor expectations that research in low- and middle-income countries (LMICs) should influence policy and practice have grown exponentially. Nevertheless, it often proves extremely difficult to meet this requirement, putting a lot of pressure on research teams.
Young Lives was set up to generate longitudinal evidence for use in shaping policy and practice to reduce poverty and inequality among children globally and in our four study countries: Ethiopia, India, Peru and Vietnam. Young Lives was developed with the understanding that policymakers and practitioners consistently use research evidence in making their decisions. Yet, during my time as the study’s Director (2005 to 2019), I found that often evidence had only a limited role. Research findings generally compete with a wide array of forces in shaping policy and practice, including received wisdom, public pressure, clientelism and policy fads.
With awareness of the difficulties involved, a number of researchers and communications experts are offering helpful guidance and advice online. Some of the most systematic tools and resources have been developed by Mark Reed, who provides insights and templates on stakeholder and public analysis, impact planning and tracking, developing a theory of change and social media strategy, and other topics. These resources highlight how research needs to be relevant and useful for potential users, which means understanding who the key stakeholders are, adapting research design and evidence to their needs and the context in which they operate, as well as delivering findings in ways that they can easily access.
While such resources are readily available to current researchers, there were very few tools to help researchers realise their impact ambitions in the early years of Young Lives. We experimented with a number of models, often learning through trial and error rather than systematic planning. Above all, it was evident that research uptake and impact do not happen just because researchers have vision, practical purpose and compelling evidence to share; nor is it enough to have a theory of change, strong ties with relevant stakeholders and/or a tailor-made policy engagement strategy. While these certainly matter, impact really hinges on two additional elements: tailoring the research design as much as possible to stakeholders’ information needs; and robust and accountable programme governance, which in the case of longitudinal research must be able to stand the test of time.
The role of research design in ensuring research impact
Longitudinal observational research can present a challenge for uptake and impact simply by virtue of its design. I took over as Young Lives’ Director prior to the study’s second survey round, i.e. before we had the data needed to conduct panel data-analysis. It soon became apparent just how difficult it can be for longitudinal research to impact policy and practice. It takes at least two, and preferably three, data rounds before correlations between children’s early life circumstances and later outcomes are detectable and policy implications identifiable. Yet policy cycles tend to be quite short and are seldom, if ever, planned to respond to this kind of long-term evidence.
The study’s initial objective was quite broad: to monitor the performance of the Millennium Development Goals (MDGs) in reducing poverty and inequality among children in low- and middle-income countries. Since 2016, Young Lives has offered a baseline for gauging accomplishments of the Sustainable Development Goals (SDGs) in relation to children and young people. To achieve these ends Young Lives tracks the predictors of, and changing relationships between, multiple dimensions of children’s development: from physical he
In time, Young Lives was able to demonstrate the policy and programmatic value of longitudinal observational data to key stakeholders both globally and in the study countries, which has been crucial to realising our impact aims. While there are a number of reasons for this, here I will focus on the contribution made by programme governance and particularly the role of partnerships.
Collaboration and partnership
The Young Lives team is comprised of research, communications, policy, data and administrative staff based at the University of Oxford and in the study-country partner institutions. As a small team with limited capacity for global outreach, the need to enhance our research uptake and impact by forging ties with intermediary organisations was apparent from the outset. In the early days, Save the Children was our policy-engagement partner and led on this work globally and in the study countries. But our research evidence didn’t always resonate with Save the Children’s objectives. The trajectory of longitudinal research is somewhat set in stone and the strength of its evidence relies on maintaining a significant degree of consistency across data rounds. By contrast, Save the Children’s information needs aligned with global objectives that over the course of the study were necessarily subject to change. Moreover, these objectives didn’t always match the evidence we generated. For example, understanding how best to reduce infant and child mortality was an important global priority for Save the Children. But the babies in the Young Lives’ sample were already aged 6 to 18 months at the first survey round and had therefore survived the highest period of risk—so we were not able to identify mortality predictors, this being a necessary step in designing preventative interventions. This led to our first major learning, while intermediary organisations are vital allies you cannot expect them to help amplify research uptake and impact unless priorities and expectations are aligned between organisations and clear from the outset.
Having joined forces in a broad policy-advocacy coalition, we now work far more effectively with Save the Children and many other child- and youth-focused agencies. Rather than being tied into long-term partnerships we do commissioned studies that respond to specific advocacy, policy and programmatic requirements and also co-convene policy events and co-publish policy-advocacy outputs with a range of agencies in our network. These institutional arrangements allow stakeholders the flexibility to use our evidence as and when required, while enabling us to provide findings tailored directly to their needs. See for example: Impact case study – Nutrition, growth and potential for recovery.
Integrating research and policy influencing infrastructures and processes
A second major learning around the role of governance in research impact has been the importance of integrating research with policy influencing work rather than running the two strands separately and in parallel. Policymakers and practitioners want to learn about research findings directly from researchers because this allows them to interrogate the evidence in detail and explore its utility for their work. So while Young Lives’ policy staff provide important context and technical knowledge on policy, undertake stakeholder mapping and the like, the researchers are also trained in research communication, policy engagement and related skills.
In terms of integrating policy-influencing work with research, hiring our four Country Directors was a game changer, revolutionizing Young Lives’ impact, because they are able to effectively bridge research, policy and practice. We recruited with great care four people who had a strong backgrounds in research relevant to Young Lives and who were well-known and had high credibility in their national policy circles. Aside from contributing considerable intellectual expertise to Young Lives’ global research agenda, in areas such as early child development, psychometric measures, national economic development and community change; the Directors are also crucial members of the senior management team. They not only coordinate and provide intellectual leadership to country-level work, but also undertake policy engagement not just nationally and sub-nationally, but globally as well.
Designing managerial procedures to enhance impact
While governance structures do matter a great deal for impact, getting institutional processes right is also vital. Having settled on our poverty and inequality reduction objectives we had to plan a way of ensuring that our inputs—i.e. resources, procedures, structures and activities—and our outputs—i.e. publications, presentations, social media products etc. —would achieve these objectives. As a comparative study we needed to gauge whether, and how, to prioritise engagement with national and/or international policymakers, advocates and practitioners. We also needed to distinguish short-term from longer-term outcomes. A short-term outcome could be, for example, when the Ethiopian Government built Young Lives evidence and advice into their plans to roll out a ‘0’ class in primary schools. While a long-term outcome would, hopefully, be the improvement to the children’s self-confidence, school attainment and their employment options in adulthood through attendance of these classes. This planning process can be very complicated, especially since research is less likely than interventions to have direct impact on development processes. Frustratingly, where research does have impact, attribution is often hard to establish.
This is where institutional and managerial procedures come in. We are required by our chief donor, the British government’s Department for International Development (DFID), to develop a logical framework (or Logframe) as the study’s core management tool. While Logframes are not universally popular, we have found them to be extremely useful in steering our work over the years. For example, we used the Young Lives’ logframe in a value for money exercise in which we compared the quality, effectiveness and cost-effectiveness of our inputs and outputs, set against our outcomes and impact, with other large LMIC research programmes. Most recently, it was used in an independent evaluation of Young Lives. These exercises have shown me that, far from being the imposition by donors many imagine them to be, logframes and other managerial tools and procedures can be very helpful in highlighting the effectiveness of research in achieving impact and also identifying where improvements are needed.
In my next upcoming blog I will share details of the Young Lives’ logframe and outline its application.
This reflection is part of a blog series for the Methodological lessons and learning in longitudinal research project. Over coming months, Young Lives will share reflections on its methodology captured in interviews, blogs, videos, podcasts and reports designed to support building a community of practice in longitudinal research in low- and middle income countries, and open a dialogue. We welcome your feedback and your involvement.
Since Young Lives was founded in 2001, donor expectations that research in low- and middle-income countries (LMICs) should influence policy and practice have grown exponentially. Nevertheless, it often proves extremely difficult to meet this requirement, putting a lot of pressure on research teams.
Young Lives was set up to generate longitudinal evidence for use in shaping policy and practice to reduce poverty and inequality among children globally and in our four study countries: Ethiopia, India, Peru and Vietnam. Young Lives was developed with the understanding that policymakers and practitioners consistently use research evidence in making their decisions. Yet, during my time as the study’s Director (2005 to 2019), I found that often evidence had only a limited role. Research findings generally compete with a wide array of forces in shaping policy and practice, including received wisdom, public pressure, clientelism and policy fads.
With awareness of the difficulties involved, a number of researchers and communications experts are offering helpful guidance and advice online. Some of the most systematic tools and resources have been developed by Mark Reed, who provides insights and templates on stakeholder and public analysis, impact planning and tracking, developing a theory of change and social media strategy, and other topics. These resources highlight how research needs to be relevant and useful for potential users, which means understanding who the key stakeholders are, adapting research design and evidence to their needs and the context in which they operate, as well as delivering findings in ways that they can easily access.
While such resources are readily available to current researchers, there were very few tools to help researchers realise their impact ambitions in the early years of Young Lives. We experimented with a number of models, often learning through trial and error rather than systematic planning. Above all, it was evident that research uptake and impact do not happen just because researchers have vision, practical purpose and compelling evidence to share; nor is it enough to have a theory of change, strong ties with relevant stakeholders and/or a tailor-made policy engagement strategy. While these certainly matter, impact really hinges on two additional elements: tailoring the research design as much as possible to stakeholders’ information needs; and robust and accountable programme governance, which in the case of longitudinal research must be able to stand the test of time.
The role of research design in ensuring research impact
Longitudinal observational research can present a challenge for uptake and impact simply by virtue of its design. I took over as Young Lives’ Director prior to the study’s second survey round, i.e. before we had the data needed to conduct panel data-analysis. It soon became apparent just how difficult it can be for longitudinal research to impact policy and practice. It takes at least two, and preferably three, data rounds before correlations between children’s early life circumstances and later outcomes are detectable and policy implications identifiable. Yet policy cycles tend to be quite short and are seldom, if ever, planned to respond to this kind of long-term evidence.
The study’s initial objective was quite broad: to monitor the performance of the Millennium Development Goals (MDGs) in reducing poverty and inequality among children in low- and middle-income countries. Since 2016, Young Lives has offered a baseline for gauging accomplishments of the Sustainable Development Goals (SDGs) in relation to children and young people. To achieve these ends Young Lives tracks the predictors of, and changing relationships between, multiple dimensions of children’s development: from physical he
In time, Young Lives was able to demonstrate the policy and programmatic value of longitudinal observational data to key stakeholders both globally and in the study countries, which has been crucial to realising our impact aims. While there are a number of reasons for this, here I will focus on the contribution made by programme governance and particularly the role of partnerships.
Collaboration and partnership
The Young Lives team is comprised of research, communications, policy, data and administrative staff based at the University of Oxford and in the study-country partner institutions. As a small team with limited capacity for global outreach, the need to enhance our research uptake and impact by forging ties with intermediary organisations was apparent from the outset. In the early days, Save the Children was our policy-engagement partner and led on this work globally and in the study countries. But our research evidence didn’t always resonate with Save the Children’s objectives. The trajectory of longitudinal research is somewhat set in stone and the strength of its evidence relies on maintaining a significant degree of consistency across data rounds. By contrast, Save the Children’s information needs aligned with global objectives that over the course of the study were necessarily subject to change. Moreover, these objectives didn’t always match the evidence we generated. For example, understanding how best to reduce infant and child mortality was an important global priority for Save the Children. But the babies in the Young Lives’ sample were already aged 6 to 18 months at the first survey round and had therefore survived the highest period of risk—so we were not able to identify mortality predictors, this being a necessary step in designing preventative interventions. This led to our first major learning, while intermediary organisations are vital allies you cannot expect them to help amplify research uptake and impact unless priorities and expectations are aligned between organisations and clear from the outset.
Having joined forces in a broad policy-advocacy coalition, we now work far more effectively with Save the Children and many other child- and youth-focused agencies. Rather than being tied into long-term partnerships we do commissioned studies that respond to specific advocacy, policy and programmatic requirements and also co-convene policy events and co-publish policy-advocacy outputs with a range of agencies in our network. These institutional arrangements allow stakeholders the flexibility to use our evidence as and when required, while enabling us to provide findings tailored directly to their needs. See for example: Impact case study – Nutrition, growth and potential for recovery.
Integrating research and policy influencing infrastructures and processes
A second major learning around the role of governance in research impact has been the importance of integrating research with policy influencing work rather than running the two strands separately and in parallel. Policymakers and practitioners want to learn about research findings directly from researchers because this allows them to interrogate the evidence in detail and explore its utility for their work. So while Young Lives’ policy staff provide important context and technical knowledge on policy, undertake stakeholder mapping and the like, the researchers are also trained in research communication, policy engagement and related skills.
In terms of integrating policy-influencing work with research, hiring our four Country Directors was a game changer, revolutionizing Young Lives’ impact, because they are able to effectively bridge research, policy and practice. We recruited with great care four people who had a strong backgrounds in research relevant to Young Lives and who were well-known and had high credibility in their national policy circles. Aside from contributing considerable intellectual expertise to Young Lives’ global research agenda, in areas such as early child development, psychometric measures, national economic development and community change; the Directors are also crucial members of the senior management team. They not only coordinate and provide intellectual leadership to country-level work, but also undertake policy engagement not just nationally and sub-nationally, but globally as well.
Designing managerial procedures to enhance impact
While governance structures do matter a great deal for impact, getting institutional processes right is also vital. Having settled on our poverty and inequality reduction objectives we had to plan a way of ensuring that our inputs—i.e. resources, procedures, structures and activities—and our outputs—i.e. publications, presentations, social media products etc. —would achieve these objectives. As a comparative study we needed to gauge whether, and how, to prioritise engagement with national and/or international policymakers, advocates and practitioners. We also needed to distinguish short-term from longer-term outcomes. A short-term outcome could be, for example, when the Ethiopian Government built Young Lives evidence and advice into their plans to roll out a ‘0’ class in primary schools. While a long-term outcome would, hopefully, be the improvement to the children’s self-confidence, school attainment and their employment options in adulthood through attendance of these classes. This planning process can be very complicated, especially since research is less likely than interventions to have direct impact on development processes. Frustratingly, where research does have impact, attribution is often hard to establish.
This is where institutional and managerial procedures come in. We are required by our chief donor, the British government’s Department for International Development (DFID), to develop a logical framework (or Logframe) as the study’s core management tool. While Logframes are not universally popular, we have found them to be extremely useful in steering our work over the years. For example, we used the Young Lives’ logframe in a value for money exercise in which we compared the quality, effectiveness and cost-effectiveness of our inputs and outputs, set against our outcomes and impact, with other large LMIC research programmes. Most recently, it was used in an independent evaluation of Young Lives. These exercises have shown me that, far from being the imposition by donors many imagine them to be, logframes and other managerial tools and procedures can be very helpful in highlighting the effectiveness of research in achieving impact and also identifying where improvements are needed.
In my next upcoming blog I will share details of the Young Lives’ logframe and outline its application.