Engaging citizens in policy-making on the internet is a relatively new practice, but one that usually comes with massive expectations. But what exactly should be measured when we speak of true success in an online participatory process? Here’s a concise guide on how to get started on it.
Start with why
Why should you measure the success of an online engagement process? An accurate measurement leads a correct judgement of the merits and flaws of the participation process. It helps you increase accountability inside the public administration, optimally manage finance and resources in the future, and assess whether the citizens’ input has made a difference in policy-making.
It is all-important to align everyone, from the technical solution provider to the members of the project team, on a handful of essential success metrics. These are often referred to as Key Performance Indicators or KPIs. Once KPIs are defined, it is time to track your progression -or regression- along trend curves that allow you to understand what works and what doesn’t. Eventually, you should be empowered to dip in at just the right time to ensure downturns don’t endure and the highs are lasting.
What to measure?
In short, one can call an online public participation trajectory a success once there is an engagement of a group of citizens representative of the broader population that has led to high-quality insights used for decision-making via a cost-efficient process.
1. Guaranteeing inclusiveness
The users you consider engaged -the crowd you actually « recruited »- must be inclusive enough. Direct online participation is designed as an improvement of our current political system, where most of the decisions occur under the approval of representatives elected by all. It is therefore natural that online participation must aim for the maximal level of representativeness among its audience. This is an imperative not only for legitimacy but for the quality of the crowdsourcing. It means large enough numbers of participants. And before a majority of the population is actually involved in your offer of a more direct democracy (which might take some time!), it is essential that the batch of joiners be diverse.
A frequent critique against digital democracy is that it only manages to attract, in essence, 25-40 year olds from the urban upper middle class with above average IT skills and formal training. Although this group may be easily converted into active participants, online participation has the potential to reach everyone. It is your responsibility to make sure that not only the digital elite get involved but that people from all walks of life take part —through improved interface accessibility, better targeted communication efforts, and so on. Hence, the advice here is to not forget carefully monitor the numbers of participants as well as their demographics.
Key measure: Participation distribution amongst pre-defined citizen groups (based on age, gender, area of residence, etc.).
2. Generating qualitative content for policy-making
There is no real point in involving people if what they share isn’t of use to anyone. Hence proposals and opinions sourced from the public should be qualitative, which entails several things. First the representativeness of your audience is key : views expressed exclusively by one segment of the opinion spectrum are no collective intelligence but rather a takeover by vested interests. Then the contributions should be addressing current and actual issues faced by the community. That is, they must be relevant in order to deserve a share of public resources or of the citizens’ time and efforts. These are limited and are always channelled to the most pressing shortcomings. Lastly but importantly, digital participation must be a vector for innovation. On top of increased legitimacy, it is expected that doubling up the government official responsible for a decision with a crowd of citizen-experts paves the way to disruptive solutions -not to dull ideas sustaining the status quo.
Key measure: Number of proposals that have actually contributed to the actual decision(s). (Note that demonstrating causal links between participation and policy outcomes is often difficult because of the time lag between processes and policy or public action. That’s why you’d better add qualitative measures to come to a more insightful case study analysis.)
3. Ensuring the process is cost-efficient
Direct online participation is here for the long run ; this means it has to be sustainable. The time and efforts required to make your engagement a success shouldn’t be disproportionate. The easiest way to measure this is to make sure the cost per engagement action remains moderate. By measuring the efficiency, you want to see questions answered such as: “Does online participation save money by easing the implementation of policies, for instance by decreasing conflicts to the final decision?” and “How much time did it take for the staff to go from design of the process over moderation to conclusions?”.
Key measure: Cost per engaged citizen, ie. the sum of monetary resources spent on the whole participation process (amongst others staff salary, communication resources, and purchase of technologies) divided by the number of engaged citizens. To make sense of this KPI, compare it to offline alternatives.
Learn from your citizens for future success
A measure that is often overlooked but has proven to be extremely insightful towards future successful participation programs is the satisfaction of your citizens. Luckily, it is fairly simple to get a representative sense of how the participating citizens felt about the participation process; a satisfaction survey does the job. This survey should ask the participant amongst others about the satisfaction with actual outcomes and his/her feedback about the participatory process design. These insights will help set priorities in which changes to be made towards your next participation programme. Good luck with it!
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
The _ga cookie, installed by Google Analytics, calculates visitor, session and campaign data and also keeps track of site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognize unique visitors.
Installed by Google Analytics, _gid cookie stores information on how visitors use a website, while also creating an analytics report of the website's performance. Some of the data that are collected include the number of visitors, their source, and the pages they visit anonymously.
Set by the GDPR Cookie Consent plugin, this cookie is used to record the user consent for the cookies in the "Advertisement" category .
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
Records the default button state of the corresponding category & the status of CCPA. It works only in coordination with the primary cookie.
No description available.
No description available.
No description available.
This cookie is native to PHP applications. The cookie is used to store and identify a users' unique session ID for the purpose of managing user session on the website. The cookie is a session cookies and is deleted when all the browser windows are closed.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
This cookie is set by Google Analytics and contains a token that can be used to retrieve a Client ID from AMP Client ID service. Other possible values indicate opt-out, inflight request or an error retrieving a Client ID from AMP Client ID service.