EEA catalogue of European environment and climate policy evaluations
The EEA catalogue of policy evaluations in the field of environment and climate policies is a database that contains extensive information on almost 600 published evaluations.
The aim of the catalogue is to provide an easy access to the available evaluations and to obtain an overview of policy evaluation practices in Europe, in the areas of environment and climate policy.
The policy evaluations included in the database have been documented according to a series of features, in order to obtain an overview of policy evaluation practices. The main categories of features include: basic information on the evaluation; policy areas and economic sectors that are covered; information on the geographic and timely scope; information on authors and commissioners; evaluation criteria used; evaluation methods and techniques applied; data used and results of the evaluation.
More information available in the methodology section and in the report (the final report can be downloaded here).
Methodology behind the catalogue
The catalogue has been developed via extensive desk research and filtering of available literature. The first step of the search process was a systematic scan of organisations’ websites who are active in publishing policy evaluations and various publication portals. For national evaluations, the first source was the EEA’s Policies and Measures (PaMs) database http://pam.apps.eea.europa.eu/ where existing evaluations on climate mitigation policies should be reported. Further, on the national level, organisations such as national ministries that work in the field of environment and climate policy, environmental or energy agencies as well as NGOs were checked. Horizon 2020 projects were looked at as well. At EU level, the most important source for identifying relevant evaluations was the European Commission, notably the evaluation calendar of the Commission. In addition, websites of relevant Directorate-Generals have been screened separately (such as DG Climate Action, DG Environment, DG Mare or DG Energy). Furthermore, other European or international institutions were included in this search such as the OECD, the European Court of Auditors and other EU institutions (European Parliament, European Committee of the Regions and European Economic and Social Committee). This initial scanning of relevant websites was accompanied by a targeted Google search. Some key words that were used for this Google search include: ‘policy’, ‘evaluation’, ‘ex-post’, ‘ex-ante’, ‘mid-term’, ‘progress evaluation’, ‘cost-benefit analysis’ in combination with ‘environment’, ‘climate’, ‘Ministry’, ‘assessment’, ‘impact’, ‘water’, ‘waste’, etc.In total, more than 2 000 documents were identified as potential policy evaluations. Each of these documents went through a filtering exercise to distinguish which would be included in the catalogue. The basis of selection was a set of criteria:
- The document consists of an evaluation of a specific (or set of) policy measure(s) or regulation(s):
- ‘Policy’ has been defined as a law, regulation, procedure, administrative action, incentive, programme or strategy, implemented (or planned) by governments or institutions;
- ‘Policy evaluation’ has been defined as the exercise of evaluating the quality, benefits, worth and utility of a given policy, using various research methods and evaluation practices. This is achieved by investigating the content, outcome, implementation or impact of a policy;
- Consequently, for the purpose of this catalogue, all literature that was included in the list of policy evaluations had to include both an ‘evaluation’ and a ‘policy’ element. Evaluations that looked at multiple policies or regulations were also taken into account in the analysis;
- Impact assessments of specific projects, typically infrastructure (e.g. roads) were excluded.
The catalogue was established in 2018 under the EEA Framework contract (Specific contract No 3413/B2018/EEA.57336, implementing Framework service contract No EEA/IEA/14/006). The number of ‘selected’ policy evaluations that comply with the criteria was 495. The list of ‘reserve’ evaluations that did not fully comply with the criteria (but could be of interest for future research), came to 366. Along with the creation of the catalogue in 2018, a meta-assessment of the catalogue was performed , as well as practical recommendations for the maintenance of the catalogue.1,2
In 2020 the catalogue was updated under the EEA Framework contract (Specific contract No 3413/B2020/EEA.58177, implementing Framework service contract No EEA/ACC/18/001/LOT2). The same methodology was used to achieve consistency in the type and quality of data between the evaluations. The number of ‘selected’ new policy evaluations was 101 and the ‘reserved‘ list consisted of 53 new evaluations.
A meta-assessment of the full catalogue and further recommendations are available in the report ‘Update of the EEA catalogue of available evaluations of European environment and climate policies’ (the final report can be downloaded here). The full list of policy evaluations is also available (the final list of policy evaluations can be downloaded here).
The policy evaluations included in the database have been documented according to a series of features, in order to obtain an overview of policy evaluation practices. The main categories of features include:
- Basic information on the evaluation such as title, evaluated policy, publication year;
- Policy areas and economic sectors that are covered by the evaluated policy;
- Information on the geographic and timely scope of the evaluation;
- Information on authors and commissioners of the evaluation;
- Evaluation criteria used in the evaluation;
- Evaluation methods and techniques applied in the evaluation
- Data used in the evaluations and results of the evaluation
The data is entered through open fields and choice of dropdown menus, which contains predefined categories for certain variables. The data shown in the catalogue can be filtered by different parameters using the right-hand drop-down options. The catalogue can be downloaded as a CSV file via the button located below the database.
1 Rob Williams; Irati Artola (Trinomics, 2018): Assessment of the catalogue of European environment and climate policy evaluations. Specific Contract under Framework Contract No EEA/IEA/14/006 (unpublished).
2 Rob Williams; Irati Artola (Trinomics, 2018): Recommendations on the maintenance of the catalogue of European environment and climate policy evaluations Specific Contract under Framework. Contract No EEA/IEA/14/006 (unpublished).
Application data last refreshed 11 January 2021 08:51 AM.
Version info eeacms/esbootstrap:v3.1.2 and tag version v3.1.5 on elastic-app-poleval-esapp-1.