Previous
We re a bit late with this post but here it is.
The Exchange has just turned one. In October, we submitted a report on the first year of The Exchange for the three institutional funders, TTI, TTF, and KSI.
This post focuses on the aspects of the report that address the work done since the first report was submitted earlier in the year.
This second period involved 4 phases: 1) Consolidation of the teams and changes in the participants, 2) Planning meetings – including a face-to-face meeting in Budapest for the “communications” team, 3) preparation, online presentation and reviews of the final proposals, and 4) the Jakarta Exchange. This post includes a fifth and a sixth phase: 5) producing and launching the Business Models project, and 6) deciding the location of the third meeting.
This blog post provides an overview of the work done so far. A second post will address changes in the plans and a future post will explore the recommendations put forward in the report.
Consolidation of the teams
After the meeting in Lima in late March 2014, the participants returned to their countries and began a process of discussions to refine the memberships of the teams and the focus of their projects. Two teams emerged: one focusing on communications and one focusing on performance self-assessment tools for think tanks. The business model project, which aroused some enthusiasm at the Lima Exchange, was dropped due to a lack of interest.
The process of consolidating the teams continued all the way to the second meeting in Jakarta in October. A number of challenges emerged during this period.
First, the participation of the Indonesian think tanks was put in doubt.
On the one hand, one of the participants, Riko Wahyudi (from Article 33) was involved in a motor accident and was forced to drop out of the collaboration. After a period of silence from him and his organisation we contacted Article 33 Indonesia’s director, Chitra Hariyadi. She assured us that Article 33 was interested in participating and volunteered herself and a colleague, Ermy Ardhyanti, to replace Riko. Although the team had already developed much of the proposal on their own, Ermy as able to join them in Budapest for a team meeting and quickly integrated into the group and joined the effort.
The second challenge came from the other Indonesian participant and also related to changes in members of the teams. For a number of reasons he was not able to join the Skype meetings that the communications team organised and he was away on Hadj (pilgrimage to Mecca) during June and part of July, a time when the teams were intensively engaged in preparing and refining their proposals. He also missed the Budapest meeting. By the time that the proposal was submitted in August, he had unfortunately, not been able to play any significant role in the team discussions or the preparation of the proposal.
This kind of unintended disruption is to be expected in collaborative projects. In fact, we had noted this as a risk in the first report. Especially with so many participants involved. The multi-lateral model worked well to share the burden of work and risk but, in this case, it may also have increased both.
In August, Bambang Hudayana suggested that another member of his organisation join The Exchange instead of him and introduced Ashari Cahyo Edi (Hari) as a possible member of the communications team. Hari did not, however, join this team. This would have been too disruptive for the original members who had already bonded and felt that they had been working very hard to see the proposal through to the next round. We talked to them directly about this and agreed that another solution had to be found.
Similarly, it would not have made sense for Hari to join the other team since he had no interest in this topic. And it would have been equally disruptive.
We therefore considered another option. We introduced Hari to Leandro Echt, who had expressed an interest at the Lima Exchange to work on a “Business Models” project. Since Leandro’s focus of work is the study of think tanks, taking on a second project would have been much less of a burden for him.
We, the organisers, were clear to both Hari and Leandro that they did not have to collaborate if things did not work out. We would help them to develop a draft proposal before and during the meeting in Jakarta and then we’d all assess progress afterwards. In the event, Hari, backed by Bambang and his organisation, IRE, took the opportunity and the initiative. During the week in Jakarta they very quickly developed a proposal on business models for think tanks, which has now been reviewed externally, and has been approved.
Another aspect of the consolidation of the teams relates to possible changes in affiliations of the participants. Another risk that we noted in our first report related to possible movement of individuals from one think tank to another. We asked the question: should we follow the think tank or the researcher?
In the cases of IRE and Article 33 we have followed the think tanks. In these cases, the original researchers were unable to participate yet their think tanks showed a clear interest to continue to be involved.
However, there were other cases in which the researchers wished to remain involved even though their organisational linkages were changing. Renata Dalaqua, from CEBRI, for example, became research associate at CEBRI and took up a similar post at the Getulio Vargas Foundation. It made sense to follow her not only because she expressed a desire to continue to be involved but also because the change in relationship with the think tank was one that could be expected.
The second case we faced concerned Leandro Echt. His relationship with CIPPEC also changed.
After a period of negotiations with CIPPEC, Leandro parted ways with the think tank and began working on his own. It was difficult to decide but given his involvement in two teams it became hard for us to ‘let go’. We decided, again, to follow the person.
With some support from the organisers, Leandro approach CADEP in Paraguay and ILAIPP (a network of Latin American think tanks) via one of its leading members, Grupo Faro, in Ecuador. He has agreed to work with them, as an associate and collaborator, in the projects in which he is involved. This is a unique and somewhat unconventional situation but we feel it has potential to crate further linkages between the participants of The Exchange and other organisations.
So, in both cases, the situation is stable and positive. In the case of Renata, she remains actively involved and now wears two hats: from CEBRI and from Fundação Getulio Vargas. And in the case of Leandro, there is potential for further regional collaboration.
Finally, it is worth nothing that the teams faced another challenge regarding membership and participation when Petra Rezteko from the Budapest Institute had to skip the meeting in Jakarta. She was, however, able to involve her fellow director Balazs Varadi in the process and he very was a very active and dynamic participant in the Jakarta discussions. He even wrote a blog post about it.
Planning meetings
The participants chose different approaches to plan their work. The Communications team chose to travel to Budapest to organise a face to face ‘writeshop’. This meeting, hosted by the Think Tank Fund also served as an opportunity for the team to meet Ermy from Article 33 and integrate her into the group. Stephen Yeo supported the team by traveling to Budapest and working with them on 2 out of the 3 days they met there to work on the project proposal.
The performance self-assessment team (PSATT) found that it was not possible for them to find a time when they could all meet in May or June, so they decided instead to use virtual meeting tools to coordinate their work.
In general, both teams have reported using email groups, Skype, BlueJeans, DropBox and Google Drive to collaborate. This phase turned out to be an excellent opportunity to test them and explore what worked best for them.
Presenting and reviewing the proposals
As noted above, the Exchange participants initially came together in two groups, each focused on a collaborative project. One of the groups met mostly online using Skype and email to coordinate the development of their proposal while the other decided to include a face to face meeting to finalise it.
Once the proposals were submitted in late July 2014, we sent them to external reviewers who had been provided with guidelines on the projects and how to evaluate them.
The external reviews had two objectives:
- To identify the strengths and weaknesses of the proposal as submitted
- To suggest ways in which the proposal could be modified to address any weaknesses present
The external reviewers were asked to focus on the following sections of the proposal:
- Objectives – are the objectives clearly stated, and coherent and viable?
- Expected Contribution – are the expected contributions realistic, given the think tanks involved and the challenges implied in what they want to achieve?
- Methodology – is the methodology appropriate given the objectives of the project?
- Work Plan – is the work plan realistic and ensures collaboration?
- Qualifications and Expertise of the Participants – do the participants have the expertise necessary to carry out the project successfully?
We asked the reviewers to focus on the Methodology and Work Plan, since they were less likely to have detailed information on the members of the team or their institutions. The reviewers’ comments were added to those provided to the teams earlier by Enrique Mendizabal, Vanesa Weyrauch and Stephen Yeo.
The communications project (CSCS) was reviewed by:
- Lawrence MacDonald: Lawrence is currently moving from the Center for Global Development where he was VP for Communications and Policy Outreach, to the World Resources Institute to take on the job of VP of Communications. He has written a blog post about his experience at CGD.
- Peter da Costa: Peter has been a pioneer of research communications. As an Africa-based consultant to the William and Flora Hewlett Foundation, he provides support across the portfolio of the Foundation’s Global Development and Population Program, including in relation to the Think Tank Initiative.
- Richard Darlington: Richard is Head of Media at IPPR in London. He is also a co-founder of WonkComms, a network of think tank communicators that started in the UK but has now spread around the world.
Table 1: CSCS
| Case studies on communication strategies focusing on building democracy projects (CSCS) | ||
| Abstract | ||
| The project aims to exchange information and improve the communication practices of five think tanks from different regions on a shared topic of interest: “building democracy”. A common topic that each think tank is working from different perspectives related to their context, needs and researchers interest.The project includes four components: a) each think tank will map the relevant aspects of its communication practices over the period 2012-2014; b) based on this mapping, the participants will agree on appropriate case studies to explore their communication activities; c) exchange and reflect on the case studies and lessons learnt from them; and d) learning by doing – designing a new communication activity for each think tank on a topic of shared interest. Participants will enrich their communication approaches, strategies, practices or decision-making processes, inspired by the exchange experience. | ||
| Participant | ||
| Name | Institution | Country |
| Francesca Uccelli | Institute of Peruvian Studies (IEP) | Peru |
| Nadia Dobryanska | Centre for Political and Legal Reforms (CPLR) | Ukraine |
| Radka Vicenová | Center for European and North-Atlantic Affairs (CENAA) | Slovakia |
| Chitra Hariyadi | Article 33 | Indonesia |
The reviewers of the CSCS proposal highlighted the following issues:
- The proposal was somewhat unclear on whether its objective was to innovate or just to reflect on and share communication practices: they require different methodologies.
- There was some confusion around terms such as ‘communication practices’, ‘systematization’, ‘activities’, etc. These should be more clearly defined in the proposal. This has implications for how the work will be done and may raise questions of comparability across the cohort (if the purpose of the collaboration is to compare like with like). Will the differences among institutions create difficulties for the project or is it an advantage? The same applies to ‘building democracy’ theme – the think tanks work on different issues and the proposal does not seem to have a clear definition of what ‘democracy’ means. Each context is likely to be different, so why would ‘democracy’ be the right theme for collaboration?
- It is important that the proposal spells out which communications activities will be included in the mapping, on what basis, and to what end. The discussion of mapping focuses on what has been done by the institutions rather than what they have learnt. It would be very useful to explain whether and what point in the project you will examine the future, e.g. possible changes in the goals of communication and the tools to be used.
- The criteria that will be applied to select the practice/activity/tool for the case study are not clear and should be clarified. In terms of case studies, the reviewers suggested taking a learning approach rather than a performance assessment one. They could become knowledge products, where lessons are detected and shared. If there are many things in common among case studies it might make sense to think about a final product where you compile and distill general lessons
- As with the case studies, the proposal needs to clarify how you envisage choosing a new communications activity. Finding one that is new for each of the participating institutions and relevant to the democracy building topic might be a significant challenge. Even though potential for collaboration of this type is very interesting, what would happen if you can´t find a common new activity that is meaningful for all? One way to solve this could be that each institution selects a new initiative and then the institutions share lessons about the process of deploying it.
- Are all the participating think tanks going to make equal contributions? – almost certainly not. So what are the relevant strengths and weaknesses of each of the think tanks and how will these be used / addressed in the project?
- It would be also be important to better define how the project plans to engage communications staff from the organisations and their role throughout the project.
The performance self-assessment project (Performance Self-Assessment for Think Tanks, or PSATT) was reviewed by:
- John Young: John is the founder and Head of the RAPID Programme at ODI. He also served as Deputy Director at ODI, where he oversaw significant internal reforms within the organisation.
- Dena Lomofsky: Dena is an experienced evaluator with a particular interest in policy research programmes and think tanks. She has been recently involved in an evaluation of a large regional programme led by a South African think tank.
- Ben Davies: Ben is one of the key architects of the Knowledge Sector Initiative in Indonesia. He left DFAT (Australia) to join the University of Sidney.
Table 2: PSATT
| “How are we doing?” Performance Self-Assessment for Think Tanks (PSATT) | ||
| Abstract | ||
| This collaborative project seeks to provide representatives and decision-makers of different think tanks with knowledge on organizational performance self-assessment. The participants will analyze existing, easy-to-apply, inexpensive tools that can improve their understanding about and assessment of their organizations’ performance and capacities. They will also identify, adapt and pilot a tool or one relevant technique (an existing tool or a mix of tools), adapted to their own organizational needs, the participants will develop a case study for each organization, summarizing tips and lessons learned to share among team members as well as with other think tanks interested in assessing organizational performance. This project shall provide think tanks in middle-income and developing countries of the globe with a problem-oriented review of existing performance assessment tools. The project outputs shall map and synthetize relevant performance assessment tools and present lessons of the pilot cases based on the experiences learned by participating organizations themselves. The main added value of the project is to systematically collect evidence and results of self-assessment based on participatory research and on a collaborative learning process. | ||
| Participants | ||
| Name | Institution | Country |
| Irina Guruli | Economic Policy Research Center (EPRC) | Georgia |
| Leandro Echt | Center for the Implementation of Public Policies Promoting Equity and Growth (CIPPEC) | Argentina |
| Petra Edina Reszketo | Budapest Institute for Policy Analysis (BI) | Hungary |
| Renata H. Dalaqua | Brazilian Center for International Relations (CEBRI) | Brazil |
| Adriana Arellano | Grupo FARO (GF) | Ecuador |
The reviewers of the PSATT proposal highlighted the following issues:
- It is important to clarify whether the project is about how well the organisations are doing “organisationally” (as stated in the first paragraph), implying an inward-looking focus on “organisational development”; or instead is about how to “measure the impact of their research or advocacy activities”, implying an external “impact evaluation” focus. Both are important, but require rather different tools and approaches.
- The proposal could be refined so that the ultimate objective is to identify tools that help your organisations improve the way they do business (operate more effectively/ improve organisational effectiveness).
- It would be good to include as additional context here that while the project is aimed at the TTs themselves, donors are looking to provide core funding to TTs that are “reflective” – and the tools can play a key role in helping to initiate this process. The project could in this way influence donor thinking too.
- In order to understand how the project can contribute to each participating institution, it would help if the proposal were to identify more clearly which areas your organizations wanted help with: i.e. improved advocacy, higher research quality, better understanding of context, improve strategic focus, be more selective in taking on projects, have better systems to support staff, etc.
- The project should ensure that it builds on what others are already doing. It should specify where the self-assessment tools will come from – other developing country TTs or developed country “domestic” TTs? There is more room for shared learning with TTs or programs or TTs like KSI (in Indonesia) that have tried self-assessments (albeit with mixed success). This is also important if the project hopes to add substantially to global knowledge in this area.
- There should be a clearer separation into stages (literature reviews, selection of tools for pilots, collaborative action-research, engagement and communication etc.), and clearer identification of methods for each stage, e.g. a simplified systematic review for the literature review, the use of a framework to select pilot projects, and simplified QCA for analysis of the results.
- It is important not to place too much emphasis on how easy the process will be given how cheap the tool is. The real challenge is likely to be getting buy-in for the tool from staff, and for this it is important to be clear on the process for integrating the tool into the organisation’s work.
- There is more scope for partners to review each other’s work / assessments – experience suggests that the quality of self-assessments increases dramatically with peer reviews and this offers a new opportunity for collaboration.
- The proposal should consider whether to link this effort to each institution’s strategic planning process for the following year
- The proposal suggests that significant share of budget be dedicated to paying an external moderator or other organisations to review the plans and help develop the tool. The role of moderator is crucial to introduce the tool, explain how it works, help them to share amongst themselves, and link them up with other TTs who have done self-assessments. While there is a danger that this will decrease the teams’ sense of ownership of the project, it has some advantages, e.g. it could serve as a means of quality assurance for the process. The team should be able to cover the costs of the proposed moderator by reducing the budget in other places. One possible way of doing this would be to have the mapping done as a desk-based exercise by one keen M&E person or by an external moderator, who then introduces the tools to the rest of the group.
- The proposal should consider the timing of the self-assessment, in particular to ensure that you will be able to prioritise exercise and it won’t collide with your busiest period.
- The proposal should clarify what each think tank will do: who will compile reports? who is the key author? There will be a need to agree on format of reports at the end and who does what.
- The proposal should also consider including one individual from each organization’s board, since the board (along with the director) will ultimately be the ones to benefit from the tools. At a minimum the proposal should explain how the board would be involved.
Each team participated in a conference call (11th and 12th August) with the coordinators to discuss the reviewers’ feedback. These were taken into account to be incorporated into a new version of the proposal (which took the form of a formal response from the team).
The third project, on Business Models, was reviewed by Simon Maxwell, former director of the Overseas Development Institute, after the Jakarta meeting but it is worth noting his recommendations here.
Table 3: FMAI
| A comparative study of business models for think tanks in Argentina and Indonesia (FMAI) | ||
| Abstract | ||
It is difficult to find systematised knowledge about which are the most successful business models for different contexts, especially in developing countries. The project seeks to provide think tanks with knowledge that will allow their leaders to shape their organisations’ business models to better succeed, given their contexts. This study will focus on think tanks in Indonesia and Argentina and will address the following questions:
The research will take place in Indonesia and Argentina. Four think tanks will be studied in each country. Besides producing 8 descriptive case studies, the research will generate a synthesis report that compares some common and critical aspects between both countries. The goal is to understand the underlying factors that generate such variations, why certain models work for certain types of think tanks and in certain contexts or circumstances. The deliverables will include: a blog post on the project proposal, publication of the literature review as a blog post, eight case studies, one synthesis report on lessons learned from the case studies, two blog posts on collaboration/learning experiences, two seminars: one in Argentina and one in Indonesia (with KSI partners in Jakarta and IRE partners in Yogyakarta) and a webinar to share results with the other participants in the Exchange. |
||
| Participants | ||
| Name | Institution | Country |
| Ashari (Hari) Cahyo Edi | Institute for Research and Empowerment (IRE) | Indonesia |
| Leandro Echt | Center for the Implementation of Public Policies Promoting Equity and Growth (CIPPEC) | Argentina |
Simon Maxwell had the following comments:
- A sense of why might it be that think tanks are constantly worrying about their business models. He offered some hypothesis or research questions to guide this. For example (and this is a very initial set):
- Relying on short-term contract funding makes it difficult to develop long-term programmes;
- Conversely, having long-term, programme funding makes it more difficult to be responsive in the short-term;
- Public affairs is the core mission of a think-tank, and the area of activity that distinguishes think-tanks from research institutes, but is the hardest to fund;
- Having one large, philanthropic donor runs the risk that the agenda will be set by the donor;
- Having a large number of small contracts increases the cost of doing business;
- The fact that some donors do not pay overheads at a high enough level significantly constrains the fund-raising activities of think-tanks;
- Having a specialised fund-raising and proposal writing department is an essential investment by think-tanks;
- Growth in turnover greatly helps think-tank finances, by spreading overheads more thinly, but there are diminishing returns;
- Think-tanks as business suffer serious problems when donors do not allow or encourage them to build reserves;
- Cash flow is often the biggest problem facing think-tanks;
- Think-tank finances consist of two elements, core costs and project costs: funding the cash flow of the latter can be difficult.
- There was concern that the proposal did not reflect this kind of engagement with the day-to-day issues. The text was too theoretical, and not based enough on the management realities.
- This can be corrected, provided that the right people are in the room – heads of finance and administration, as well as ‘researchers’, for example.
- A workshop to develop hypotheses, perhaps initially in each institution, would be good.
- The next question is what kind of evidence is needed to substantiate the hypotheses. Will there be case studies of individual projects or programmes? Stories of change? Statistical analysis of annual accounts? Probably some combination of all of those.
- The final point is that there probably won’t be a single answer to the problem of how to finance think-tanks. In fact, there certainly won’t be. So what’s the output? A series of possible pathways and options? An analysis of key decision points?
- It would be useful to have some Harvard-style management cases. The main thing is to think from the beginning about what would make this useful to think-tanks in the future. What you don’t want is a conclusion which simply says ‘this is a very difficult problem, every institution is different . . .’.
The Jakarta Meeting
The second meeting of The Exchange (for all the participants) took place in Jakarta, Indonesia, between 29 September and 4 October 2014. There were a number of reasons why we chose Jakarta.
Given the earlier challenges we faced in engaging with the Indonesian participants we wanted to make sure that they received our full attention and support to make up for the ‘lost time’. By holding the event in Jakarta we would be able to meet them in their home territory. We hoped that this would make them more comfortable and better able to engage effectively and integrate fully with their teams.
There was also a more practical reason for this. The Knowledge Sector Initiative is based in Jakarta and we wanted to ensure their support and participation. There was also an added benefit: KSI had invited Orazio Bellettini (Executive Director of Grupo Faro and one of the leaders of the Latin American network of think tanks, ILAIPP) to talk to its Indonesian grantees in late September. We thought it would be a good opportunity to combine both events. In the event, KSI had to cancel the event with Orazio but The Exchange helped him coordinate an alternative meeting with another Indonesian think tank, SMERU.
We contacted Article 33 Indonesia as soon as Chitra expressed an interest in remaining part of The Exchange. Since she had not been present in Lima we shared the workshop reports with her as well as some of the impressions from the participants.
Article 33 Indonesia coordinated the organisation of the Exchange with IRE and KSI. This helped to ensure that both organisations participated fully in the event.
As before, we shared the draft workshop outline with the participants and incorporated their comments. The workshop reports can be read here:
The Jakarta meeting provided ample opportunities to learn about collaboration. Day three was dedicated to this, led by a presentation by Orazio Bellettini at SMERU which involved all the Exchange participants
Jakarta also proved an opportunity for everyone to meet Ermy and Hari. IRE’s Executive Director, along with Bambang and Hari were present on the first day of the meeting to reassure the group that the organisation was committed to participate in the Exchange. Hari took the initiative at many points during the open meeting on the third day, and Ermy proved to be an outstanding host at Article 33.
Most importantly, the participants had a chance to get to get to know each other. It was clear that more personal relationships were developing since many ‘friended’ each other on Facebook and WhatsApp.
During the meeting in Jakarta, Leandro Echt and Hari worked on a draft proposal for a new project on business models. The objective of this possible third project is to compare business models of think tanks in Argentina and Indonesia.
Brokering a new relationship
A significant amount of effort was placed, during this period, to brokering a new relationship between Leandro and Hari, in the context of changed in the institutional affiliations of the participants.
This process started a couple of months before the Jakarta meeting and finally finished in late December with the final version of the project proposal being approved.
While this kind of support was somewhat expected it was also rather more complicated than anticipated. There were many aspects of the process that had to be managed:
- The technical aspects of the proposal development process;
- The softer aspects of personal introductions and communication;
- And the more diplomatic aspects related to the relationship between The Exchange and the think tanks (CIPPEC and IRE). This was further complicated by Leandro’s departure from CIPPEC.
Bidding to host The Exchange
This last component of the work undertaken during this period will be further described in another blog post but presents an interesting and positive case.
We considered that this was a turning point in the project. First, The Exchange was reaching its half way mark. Also, we were beginning the projects in ernest. Finally we had noticed how the participants had began to call the shots and organise the ‘workshop time’ in Jakarta.
We thought that it was time to let them take on the initiative. Deciding where the next meeting would be presented an opportunity. Therefore, we asked them to bid to be hosts of The Exchange.
CEBRI in Brasil and Grupo FARO in Ecuador sent bids and, after a very tight voting process, Grupo FARO was chosen as the host.