Lessons from Latin America’s First Evidence Week

More than 1000 people participated in close to 30 events dedicated to celebrating, learning about, and strengthening the role that evidence plays in public policymaking in Latin America.

The First Evidence Week was a festival of events was promoted by On Think Tanks along with the Peruvian Alliance for the Use of Evidence, the Universidad del Pacífico and the Peruvian Monitoring and Evaluation Network (REDPERUME) and was possible thanks to the initiative of many other organisations among think tanks, ministries, NGOs, universities, media houses and corporations. Rather than taking control of the organisation of all events during the #SemanadelaEvidencia, we opted for a co-production approach in which each event was produced and hosted by organisations across Lima and Peru.

What did we learn?

The point of such an event, of course, is to draw out lessons from the various speakers and participants who were involved in the events. Here I suggest a few take-away lessons that we hope to explore over the next year in preparation for the Second Evidence Week in 2017.

Policymaking is not easy

This may sound obvious but it can appear that researchers, NGOs, the media and citizens often forget how hard it is. Demands for a greater use of evidence in decision making are being accompanied by greater demand for direct citizen participation, greater engagement and responsiveness from government bodies and individual officials, forensic scrutiny by anyone with access to a blog or an ever growing ecosystem of online fora and initiatives, etc.

While government struggles to make sense of the way in which social and technological changes are affecting their business many think tanks, universities and other interest groups have leaped ahead to develop and deploy new and very effective communication and influencing competencies and skills.

While these may promise impact in the short term they may do so at the expense of long term institution building.

Rather than single-minded policy influence then think tanks ought to consider how to support other organisations in the policy research ecosystem. This involves efforts such as:

• Helping set-up government think tanks such as J-PA’L’s and IPA’s support to the Ministry of Education in Peru to establish MineduLab (an RCT focused experimental think tank);
• Supporting the capacity of government to implement policy such as GRADE’s work on FORGE with the Ministry of Education;

• Developing new concepts and new meteorological approaches such as Universidad del Pacífico’s work on multi-dimensional poverty.

Future discussions should incorporate lessons from think tanks like Involve in the UK which study public participation

There are institutional, individual and contextual drivers of change

While avoiding the simple and flawed characterisation of the policy research ecosystem as a market we can still explore what decisions and actions can boost both the demand for and supply of research based evidence.

In Peru we can identify three key drivers of this process:

1. Results based Budgeting as a guiding policymaking approach which has established a formal demand for evidence to inform programme design, monitoring reports and impact evaluations. Although it prioritises quantitative data and promotes a results based approach that we know is not always applicable to all aspects of public policy it has driven a significant change in the evidence based policy landscape. It has, too, in a short time, created a demand for evidence and in particular evaluations that Peru’s research community has not been able to meet.
2. Technocratic ministers who have attracted a new generation of young  policymakers -many of whom joined government fresh from think tanks or graduate degrees in the United States, Britain and the rest of Europe. For the most part, these ministers have moved from think tanks (such as in the cases of Environment, Inequality and Education, which have been the more high profile one in recent years) which in other cases they joined government from international bodies and the private sector. This non-political quality has presented an attractive proposition to a professional class who would not have otherwise considered joining government -hitherto associated with nasty politics, corruption, and a rejection of evidence based policy.
3. All of this has been accompanied by a boom in post graduate degrees dedicated to public policy, including several certificates focused on evidence based policy.

Progress is uneven

While great progress has been made in a number of policy areas and in relation to certain policy research skills there are significant gaps in what has been achieved.

1. There are many issues and people we know very little about. Norma Correa from Universidad Católica, argued that rather than big data, Peru needs pick data: more detailed and nuanced information about the most vulnerable groups and challenging problems -precisely those whose condition has defied more than two decades of continued economic growth. The development of research competencies and interests have so far mainly responded to funders’ own interests -and policy research funders have tended to be foreign. Recently, a more active state and an increasing willingness of foreign funders to work with Peru’s own political imperatives have help to align agendas. But many groups (and issues) have fallen through the cracks.
2. There are many research skills that have yet to be developed. The growth of Results based Budgeting as a policymaking approach has evidenced a lack of impact evaluation skills among think tanks and universities in Peru. The Ministry of Economy and Finance, that manages Results based Budgeting has had to seek-out foreign consultants to deliver evaluations on time. There is a role for think tanks with these skills to share them with others. This is not a matter of strengthening the competition but rather of strengthening the sector: all boats rise.
3. Evidence based policymaking skills are still unevenly distributed. While there are many excellent policy analysts in the Ministry of Economy and Finance, the Central Bank and, increasingly, at the top of several ministries (education, inequality, environment, health, and production, for instance), Congress, implementing levels of ministries, and regional and district level government are all but devoid of these skills. The case of regional and district level government is of particular concern as hundreds of millions of dollars from mining taxes have been languishing in their back accounts for well over a decade in the hope that good policymaking will find a way to unleash them on public works and services.
4. Research capacity is all but centralised in Lima. The best universities and think tanks are based in Lima. Consultancy firms, of the kind that undertake research and advice governments, are based in Lima. Most universities outside Lima have specialised in teaching and underinvested in research capacity. This works well for Lima based centres who can attract funding to address challenges that would be better handled locally. But it does not work well for local governments and populations who need local support and advice.
5. Class, ethnic, and gender inequalities are as prevalent as geographic inequalities. Gender in particular emerged during the Evidence Week as a dominant concern for researchers (predominantly female researchers). Over the last couple of years several initiatives to support women in research have emerged: Grupo Sofia, Plataforma Comadres, and Eventos para Hombres, to mention three.+ We heard that women face unacceptable behaviours from their subjects including policymakers (when they approach them to offer advice or to undertake research about their institutions or sectors) and, more concerning, from their own team members. To address this they often opt-out from policy sectors that are male dominated or associated with men, adopt more “masculine” approaches to their own work (from theoretical approaches to research methods to their own professional behaviour), or simply drop out of research duties all-together, preferring instead to take on administrative tasks or increasingly leave research altogether.

The “business models” for policy and research need to change

The first thing that comes to mind with this statement is the need to reconsider the business models of research centres (think tanks and universities alike). The typical model combines relatively modest base salaries with unlimited flexible income from research projects and consultancies. In the case of many independent think tanks, researchers are not even employed by the organisations but are instead members of their boards or councils.

While this is great for think tanks facing unexpected income shocks (this means that the staff burden is low) it can be quite damaging for women’s careers. It awards greater benefits for men who have more disposable time to spend on research and consultancy projects and does not always provide for maternity cover or other medical benefits in the cases where researchers are independent contractors rather than full-time staff.

Worse still, the model of recruitment is highly dependent on personal and professional networks which leads to uneven, to say the least, relations of power between senior researchers and younger research assistants.

This closed and opaque approach to recruiting and the general business model have effects on collaboration between researchers from different think tanks and universities (this is very limited). There are few opportunities to move from one think tank or university to another.

This relatively constrained community is mirrored in government where appointments to senior positions (those driving many of the new evidence based policies and innovations) are also governed by closed personal and professional networks of patronage. They may create direct lines between think tanks and ministries but these are often exclusive of other relationships.

This is reflected in other aspect of government, too. Significantly, speakers at the Evidence Week’s Inaugural Panel agreed that the government systematically fails to coordinate across bodies (horizontal and vertically) and share information and knowledge. This approach to government throws in the face of open government developments but also undermines efforts by the government itself to invest in the generation of evidence: if it is not shared it cannot be used.

Research and policy bodies must embrace a new culture of collaboration in planning, in evidence generation, in sharing evidence and knowledge, and in the use of evidence. This is a necessary condition to achieve any significant progress towards institutionalising the use of evidence.