Previous
This blog is part of a webinar series, which shares lessons from OTT’s work as part of a project providing technical advisory support to EdLabs work to strengthen evidence use in education policy funded by the Jacobs Foundation. In collaboration with four EdLabs in Côte d’Ivoire, Ghana, Switzerland, and Colombia, Jacobs Foundation is working on enhancing evidence use in education policy and practice. This blog series features analysis and commentary from both OTT internal and external experts.
In today’s fast-paced policy environment, the ability to access and utilise high-quality evidence is crucial for informed decision-making. Repositories serve as a centralised hub for gathering, organising, and accessing research evidence, allowing EdLabs to make evidence-informed decisions that contribute to the culture of evidence use in education. Repositories are useful tools for EdLabs in supporting evidence-informed methods, sharing knowledge, encouraging collaboration, influencing policy, and evaluating their programs. They also serve as a go-to for policymakers who need evidence to make informed decisions.
The ability of an EdLab to have such a resource readily available for policymakers increases the potential for policy impact. However, deciding to design a repository requires careful consideration. As seen in the evaluation of the DFID online research portals and repositories including Eldis, R4D, SciDev.Net, and GDNet, simply creating repositories is not enough. Despite significant investments in their design and management, user uptake remains low, with many policy actors defaulting to familiar tools like Google.
This disconnect highlights a critical challenge for entities like Edlabs, which are dedicated to advancing policy impact through evidence-informed solutions: how to design repositories that truly meet the needs of busy policymakers.
Given the complexity and variety of repositories, EdLabs must carefully consider multiple factors when designing and managing these platforms, policy labs should prudently consider the design, usability, accessibility, and sustainability when thinking of designing and managing repositories. Does the team have the capacity to implement and maintain the repository? Are there enough financial resources to manage the repository? What are the data protection regulations for storing information? Who are the intended users? How will users access evidence? Are there other repositories in place? How will you ensure constant feedback on usability, and how will users know about it?
These and other questions were explored in depth during a webinar hosted by OTT in June 2024. Evidencia Midis, a knowledge translation initiative led by the Peruvian Ministry of Development and Social Inclusion (MIDIS) since 2017, shared their experience in designing and managing the repository. In this blog, we share their reflections and the key lessons emerging from the webinar discussions on building effective policy research repositories. From ensuring political buy-in to prioritizing user-centric design, these insights are essential for any organization, including Edlabs, looking to create repositories that not only store information but actively facilitate evidence use in policymaking.
What are Repositories?
Repositories are databases or platforms that store and provide access to information and resources pertinent to policy development, implementation, and evaluation. These platforms compile, categorize, and assess the quality of evidence, making it accessible for informed decision-making. Whether it is aggregating research findings like the What Works Clearinghouse, providing access to statistical data as World Bank data does, or storing evaluations like the Global Development Network (GDN) and 3ie Impact Evaluation, each repository serves a unique purpose. Even though they all provide evidence, they have varying definitions of ‘quality’ and what evidence should count in decision-making.
It is important to note that repositories are more than just digital storage systems; they are essential tools for supporting policy development, implementation, and evaluation.
Key Lessons for Effective Policy Research Repositories
Lesson 1: Repositories need political & institutional buy-in
Building a repository is just the first step; ensuring its long-term success requires the support of policymakers, institutions, and other stakeholders. Studies by Trotter et al. (2014) highlight the pitfalls of “piecemeal” approaches, where donor-funded repositories lack long-term support and local ownership.
Creating a supportive policy framework and encouraging use is crucial. Evidencia Midis, Peru’s innovative repository for policymakers, is an example of a repository that overcame this challenge. Midis ensured the support and engagement of current and former ministers of development and social inclusion by making the Ministry’s technical leadership of the DSGE see value in developing and strengthening MIDIS’ ability to produce and disseminate evidence. Currently, MIDIS and the DSGE have internal norms and frameworks. This support has been instrumental in establishing internal norms and frameworks that foster a culture of evidence creation and use within the Ministry.
Lesson 2: User-First Design is Key
Policymakers are often pressed for time, making usability a critical factor in the success of a repository. Unfortunately, repositories designed primarily by researchers can be difficult to navigate. As Fourie (2018) notes, searching through most repositories can be an arduous task. The DFID evaluation found policymakers opting for speedy Google searches over the platforms’ internal search functions. Additionally, local policymakers are often mandated to consult local official sources due to incentives or legal frameworks. Langer et al. (2016) recommend applying information design principles to improve user experience, ensuring that repositories are not just comprehensive but also accessible and easy to use.
Lesson 3: Building Trust and Credibility
Policymakers rely on trusted sources to make informed decisions. Despite government support, the What Works Clearinghouse faced criticism for misleading conclusions in its reviews, which harmed its reputation as a reliable resource. Building trust requires quality, consistency, transparency, and showcasing the value of the repository for policymakers. Testimonials, endorsements, and user reviews can help to establish and reinforce credibility.
Lesson 4: Resources, both Human and Financial, are Crucial
Creating a locally hosted repository offers flexibility and control but also demands significant resources. Scalability poses challenges, and a budget needs to be allocated for storage and maintenance. Dedicated staff are essential for data security, backups, and user support. In resource-constrained environments, a “minimalist” approach may be more feasible, focusing on reducing dependencies on network data storage while still meeting core needs.
Lesson 5: Always get feedback
Continuous improvement is essential for any repository to remain relevant and useful. Regular user feedback ensures that the repository evolves to meet the changing needs of its users. Key questions include:
- Are users finding the information they need?
- Is the search function efficient?
Gathering feedback helps tailor the repository to user needs as Fourie (2018) notes that searching through most repositories can be arduous to serve users’ needs.
Conclusion
Repositories are not a one-size-fits-all solution. Policy labs and other actors considering creating a repository need to have a clear understanding of why they want a repository, with careful consideration of the users’ needs and the sustainability of the lab. Importantly, repositories are not always the best option for every situation. There are other ways to ensure that evidence reaches the final users, and context matters greatly in assessing whether a repository is the right solution or if another approach is needed. There is ample space to study and test different alternatives to repositories, and Edlabs are uniquely positioned to explore and implement other solutions that improve access to evidence.
Effective policy research repositories are more than just digital storage spaces; they are dynamic tools that can significantly influence policy decisions. By prioritising user needs, fostering trust, and securing resources, we can create repositories that not only store information but actively empower policymakers to make evidence-informed decisions. Ultimately, the success of a repository depends on its ability to connect with its users, meet their needs, and sustain their engagement over time.
Tell us what you think here.