Caroline Cassidy writes about a novel approach to measuring the impact of public events. The post provides practical advice on how to design and use M&E tools for events. She argues that since not all events have the same objectives, not all events should be evaluated using the same indicators.
Posts tagged ‘events’
Public events are an excellent communication and convening tool for think tanks but few use them to their full potential. This post outlines some advice on how to produce an event for impact. It argues that events can be cheap and entertaining -for speakers and audiences alike- but they have to be produced more carefully.
Should think tanks charge for content as an approach to increase and diversify their revenue sources? Maybe -or maybe not. But it might be worth exploring what models are out there and how they could be adapted for think tanks. One in particular, the live-gig model offers them a chance to raise income and reach key audiences.
This month, Policy Exchange, a think tank in the UK, is hosting another of its policy fight clubs: Policy Fight Club – In the age of austerity we cannot afford to ring fence the international aid budget.
The format is quite fun and interesting. It is very different from the usual events that think tanks in the aid industry and, let’s be honest, in many developing countries organise: to begin with, not every one agrees with each other.
Disagreement is expected and encouraged and the debates that ensue effectively bring evidence and politics together in ways that publications, fancy websites, and other communication channels rarely can.
You can watch past events here but some examples below:
- Policy Fight Club – It’s time for an independent Scotland?
- Policy Fight Club – Drugs: Time to Legalise?
Do you know of any other organisations that organise similar events? Do let us know.
Micah Zenko’s post on How to Attend a Talk: Etiquette for Students, Wonks, and Speakers is a bit tongue-in-cheek. We’ve all seen them:
The “What About My Thing?” Person: This person begins their comment with the phrase: “I was wondering if you had considered…?” But, of course, the speaker hadn’t, because the contribution has nothing to do with topic presented.
The Shoehorn: Similar to the previous type, this Jenny or Johnny One-Note interjects their pet peeve, no matter how entirely disconnected it is to the topic at hand. This can be expected from activists of every ideological stripe, staunch nationalists, or fundamentalists of any faith or methodological approach.
The Wonk Hipster: The wonk hipster initiates the unveiling of his hidden gems with, “Well if you’d bothered to read,” or, “You mean you’ve never heard of…?” More often than not, the secret information will not even be in the public domain, such as a recently assembled dataset that hasn’t appeared in a peer-reviewed journal.
The “You Don’t Know Me!” Foreigner or Ex-Pat: This person was either born or spent many years living in a country under discussion. They need to emphasize that “their” country is very complex, with a rich and complicated history, and one that is wholly impossible for outsiders to ever comprehend.
The Spoiler: At events, this audience member is often a respected professor or elder statesman, who has marshaled a team of graduate students and researchers to advance and defend their cause. In addition, the spoiler is often at odds with someone else in the audience, or with a wider academic or policy field, and uses their time to loudly berate some marginal aspect of the speaker’s presentation, blithely unaware of how jarring and inappropriate it is to the discussion.
The Conspiracy Theorist: Like Occam’s razor inverted, the complex logic chain behind their comments—for the conspiracy theorist never asks a question—requires an unbelievable level of secret cooperation between governments, businesses, the media, unions, and/or fraternal organizations. Other members in the audience, socialized to be tolerant of diverse viewpoints, allow the conspiracy theorist to ramble far too long.
Zenko advises that:
The next time you are sitting in such an event, ask a relevant question, pass the microphone, and listen.
I am not sure I agree. I have felt, for some time, that the usual approach to events in many think tanks (listen, ask a question, listen) is not just not-good-enough but also contradictory of the whole idea of a think tank. All too often I’ve been to events where participants are warned to limit themselves to questions. No comments. There is an assumption that the speakers are the experts (the only experts) at the event. But the event themselves are not organised to encourage debate. Often there are one or two speakers and one or two discussants but the latter are not there to offer an alternative view and rather they use their time to support the speaker’s conclusions or messages.
Events, the truth is, are seen as just another communication channel for think tanks rather than an opportunity to encourage a discussion and defend their own ideas in open debate. The Battle of Ideas offers an alternative:
No doubt many debates at the festival will be contentious. Indeed we invite speakers and audience alike to challenge conventional thinking. Of course sensibilities will be offended over the weekend. But let’s hold our nerve; after all, as the name suggests, the Battle of Ideas is not afraid of dissenting opinions, and encourages people to speak their minds.
it would be great if events had more than a single view, if the guests did not always agreed with each other, and if participants were allowed to voice their opinions (no rants, please) -and treated as equals. It may take a while but think tanks can do a great deal by educating their audiences.
As Jeffrey Puryear argued, the most interesting and valuable contributions of think tanks may not be intellectual but psychological: the hundreds of events and seminars that Chilean think tanks organised throughout the 1980s helped to restore the mutual trust and understanding missing from Chilean politics and that had led to the rupture of democratic order.
Learning how to develop and use arguments, how to adapt them, communicate to different audiences, incorporate new ideas, defend one’s own beliefs, etc. are not matters that should be quickly dismissed. How we communicate is as (if not more) important than what we say.
Responding to digital disruption of traditional communications: three planks to ODI’s digital strategy
This first blog in a series on digital strategy and how it has developed at ODI introduces key issues the strategy is responding to, and why it is so important for a think tank like ODI.
To go public or not to go public… How to decide if it is time to adopt a new communication approach?
Many think tanks face the difficult decision over what to do about these new ways of communicating that seem to be all the rage with their funders and other competing organisations; media exposure, more accessible types of publications, public events and online communications may seem normal for some think tanks but for others they represent a significant departure from business as usual.
And this business, they feel, is doing alright; so why change it? For years (in some cases for decades) these organisations have successfully influence policy (domestic and international) by following a rather simple but effective approach: good research, good business management, and good relations with key decision-makers.
Now they are facing demand (from donors and sometimes from within) to change their approach. The pressure from outside is often motivated by donors’ branding and visibility requirements -and have little or nothing to do with what is needed to inform policymaking more effectively.
But sometimes, pressure comes from within -and for quite important reasons. Mrs Nuning Akhmadi (from SMERU, Indonesia) put it perfectly when she said:
How can we maintain the good relations we have with policymakers and still be accountable to the public?
Under pressure from funders, some cave in but others are still resisting -unsure about what to do but aware that their responsibility to the public must be met.
My advice is that before making a decision like this -which may involve a large investment and long and difficult change- you should attempt to answer two questions:
- Has the world changed? This is an important question and although easy to dismiss as obvious it may require more attention than often given. The world is not your world -made up of your think tank’s immediate sphere of influence. The reason why this doubt has emerged is that your world has not changed much -or at least that is what it feels like. It is likely that your publics, audiences and clients have not changed, that your funders (national or international) are the same, that the policymakers you’ve been working with are old friends or acquaintances, etc. But the world around you may have changed nonetheless. The media may be reporting on the issues you work on more actively thus attracting more public interest on them, social networks may be developing in your country enabling the general public to engage in policymaking in ways they had not before, and reliance on online sources of evidence may be increasing. More fundamentally, the way people learn, online and socially, may change the knowledge sector entirely. Even worse, new and better prepared think tanks (and other organisations) and new policymakers may be competing more aggressively in this new world.
Of course it is possible that the world has not changed -but is it changing? This is equally relevant. Are new generations of policymakers not used to your think tanks and modus operandi making their way up the power ladder -do they even know who you are? Are donors staffed with advisors who know less and less about your country and organisation? Are new donors paying attention to your organisation?
In a new world, the way you used to do things -most likely in private- will not work any more. If the world has changed then your audiences and funders will be looking for public debates, transparent research processes, engagement rather than dissemination, different types of outputs, multiple communication and learning channels, etc.
- If the world has changed, the second question to answer is: what should we do about this -how far and how fast should we change? Whenever someone tells me that they are thinking of setting up a blog and ask for my view I say: don’t do it. Any new way of communicating (before, during and after the research process) demands careful planning. The way you communicate, internally and externally, defines who you are -and how others perceive you. There is a good reason why this is such a difficult issue to address: going public is not just about sending a few press releases or posting semi-complete ideas online, but rather about redefining the role that your organisation seeks to play in society. Using the media to communicate says that you are accountable to the public -and not just your funders or a few policymakers; publishing online and using social networks says that you encourage debate and feedback from anyone -and not just your ‘expert’ peers.
In some cases I would not advice such radical changes -some think tanks may be better off, even in a new context, keeping a low profile and working through others.
Many of these changes involve letting go -giving up control of the research and communication process- and so you must be first confident of your position and functions.
The point is that you need to decide what kind of change is the most appropriate for your own organisation -and much of this will come from answering the first question. And unavoidable trial and error.
Both questions are researchable questions. Both merit time and resources -and, if they are serious about it, your funders ought to be willing to cough up the funds. Don’t just rely on communication ‘experts’ (or even advice from this blog) -after all, what else could they suggest if not to do what they know best? Attempt to answer these questions yourselves. Others can help but the final decision ought to be yours.
Wouldn’t it be quite ironic if they forced you (a think tank) to make a decision without the evidence to back it up?
There are some interesting examples (or attempts) of this type of research that may be worth looking into:
- A couple of years ago I edited a book on think tanks and political parties in Latin America with Kristen Sample that sought to better explain the changes in the political context of think tanks.
- This year I am working on another edited book with Norma Correa on the political economy of research uptake -it will be coming out in August 2011.
- CIPPEC in Argentina (and regional facilitators of the EBPDN), led by Vanesa Weyrauch, has undertaken a series of case studies on the way in which think tanks attempt to influence policy, a recent study was edited by Carlos Acuña. This work has contributed to make this a researchable subject.
- Harry Jones and I worked on a study on how DFID makes decisions to find out the way in which research and evaluations are used by the organisation. I came up with the approach, which Harry then used for the Asian Development Bank, after reading Gary Klein’s work on decision making. Unlike case studies, that track forward the influence of research (and consequently overestimate its importance), this approach allowed us to consider how the world of DFID worked and any possible trends in the manner in which knowledge was being used.
- Following this approach but applied to national discourses, Emma Broadbent is undertaking a study for the EBPDN on the political economy of four policy debates (HIV policy in Uganda, GMO policy in Zambia, downtown Accra decongestion in Ghana, and the chieftaincy reform in Sierra Leone) to assess the relative roles that research has played in each case -and the specific roles that think tanks may have been able to perform. Her work will be coming out later in the year but email her if you’d like to know more about it.
- Finally, a particularly interesting study (and that may be relevant for many think tanks) is this reflection undertaken by Frances Cleaver on the challenges she and her colleagues faced in their attempts to communicate complex water management messages to DFID -and others.