EUROPEAN COMMISSION

Brussels, 17.12.2024

SWD(2024) 279 final

COMMISSION STAFF WORKING DOCUMENT

Accompanying the document

Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee, the Committee of the Regions and to the European Court of Auditors

Interim evaluation under Regulation (EU) 2021/785 of the European Parliament and of the Council of 29 April 2021 establishing the Union Anti-Fraud Programme and repealing Regulation (EU) No 250/2014

{COM(2024) 572 final}

Table of contents

1.Introduction

1.1.    Purpose of the evaluation    

1.2.    The scope of the evaluation    

1.3.    The methodology    

2.What was the expected outcome of the intervention?

2.1.    The programme and its rationale, its objectives, its components and their rationale    

2.1.1    The programme and its rationale    

2.1.2    The programme’s general and specific objectives – context    

2.2.    Points of comparison    

2.2.1    Situation before the start of the intervention    

2.2.2    Overview of the points of comparison by evaluation criterion    

3.How has the situation evolved over the evaluation period?

3.1.    Implementation of the programme    

3.2    The funding of the programme’s components and actions    

3.2.1    Budget and budget implementation    

3.2.2    Annual work programmes (AWP)    

3.2.3    Monitoring and reporting method    

3.2.4    Actions under the Hercule component – analysis    

3.3    Association agreement between the EU and Ukraine on participation    

3.3.1    Scope of the association    

3.3.2    Terms and conditions of participation in the Union anti-fraud programme (excerpts of the related Commission Decision)    

3.4    Eligible actions of the UAFP – Hercule component    

3.5    Eligible entities – Hercule component    

3.6    Implementation of the Irregularity Management System (IMS) and the Anti-Fraud Information System (AFIS)    

3.6.1    Implementation – 2023    

3.6.2    Main activities    

3.6.3    IMS – budgetary implementation    

3.6.4    Maintenance and development    

3.7    IMS developments    

3.8    The Anti-Fraud Information System – Implementation    

3.9    Implementation of the UAFP in 2023    

3.10    AFIS – developments in the past decade    

3.11    AFIS budget evolution between 2014 and 2023    

4Evaluation findings (analytical part)

4.1.    To what extent was the intervention successful and why?    

4.2    How did the EU intervention make a difference and to whom?    

4.3    Is the intervention still relevant?    

5What are the conclusions and lessons learnt?

5.1    Conclusions    

5.2    Lessons learnt    

Annex I: Procedural Information

Annex II. Methodology and Analytical models used

1.    Principles    

2.    Study tasks    

3.    Data collection tools    

4.    Desk research    

5.    Documentation analysis    

6.    Stakeholder consultations    

7.    Case studies    

8.    Limitations to methodological approach and robustness of findings    

Annex III. Evaluation matrix and, where relevant, Details on answers to the evaluation questions (by criterion)

1.    The evaluation matrix    

2.    Details on the evaluation method and matrix    

3.    Answers to the evaluation questions    

3.1 Effectiveness    

3.1.1 Main conclusions: effectiveness of the UAFP    

3.1.2 To what extent have the training actions and technical assistance funded under the UAFP been effective in improving the activities linked to prevention, detection, and investigation of fraud and other illegal activities detrimental to the EU’s financial interests?    

3.1.3 To what extent have the activities funded by the UAFP contributed to enhanced transnational cooperation?    

3.1.4 To what extent have the technical assistance interventions contributed to a multidisciplinary cooperation?    

3.1.5 What factors influenced the effective implementation of the technical and operational activities funded under the first component of the UAFP budget?    

3.1.6 How did technical assistance activities contribute to limiting the currently known exposure of the financial interests of the Union to fraud, corruption, or other illegal activities?    

3.2 Efficiency    

3.2.1 Main conclusions: efficiency of the UAFP    

3.2.2 To what extent have the desired effects been achieved at reasonable costs on the basis of a cost/benefits analysis?    

3.2.3 To what extent has the UAFP facilitated a more integrated use and simplified management of financial resources?    

3.2.4 To what extent has your UAFP intervention led to any cost savings in your work related to the prevention, detection, and investigation of fraud and other illegal activities detrimental to the EU’s financial interests?    

3.2.5 Budgetary flexibility    

3.3 Coherence    

3.3.1 Main conclusions: coherence of the UAFP    

3.3.2 To what extent are the three components and the different types of actions of the programme coherent among each other?    

3.3.3 To what extent are the interventions of the programme coherent with other measures or actions taken at EU level by the Commission or institutions, bodies, agencies, which also may have contributed to the protection of the Union’s financial interests?    

3.3.4 Coherence with EU horizontal policies    

3.3.5 Coherence with other instruments and interventions    

3.4 Relevance    

3.4.1 Main conclusions: relevance of the UAFP    

3.4.2 To what extent have the specific objectives been relevant to the general objectives in the period 2021-2024?    

3.4.3 To what extent are the specific objectives still relevant to emerging trends, including in the development of new crime types and techniques?    

3.4.4 To what extent have the activities of the programme been relevant for achieving its specific objectives in the period 2021-2024?    

3.5 EU added value    

3.5.1 Main conclusions: EU added value of the UAFP    

3.5.2 Does the intervention at EU level provide added value in terms of the efficient use of financial resources as compared to a possible intervention at national level?    

Annex IV. Overview of benefits and costs - simplification and burden reduction

1.    Hercule component    

1.1.    Cost items    

1.2.    Benefits    

1.3.    Comparison of costs and benefits    

2.    AFIS component – costs and benefits    

2.1.    Cost items    

2.2.    Benefits    

2.3.    Comparison of costs and benefits    

3.    IMS component – costs and benefits    

3.1.    Cost items    

3.2.    Benefits    

3.3.    Comparison of costs and benefits    

Annex V. Stakeholders consultation - Synopsis report

1.    Introduction and stakeholder consultation strategy    

2.    Approach to the consultation    

3.    Online surveys    

4.    Overview of survey respondents    

5.    Targeted interviews    

6.    Overview of interview participants    

7.    Main stakeholder feedback per evaluation question    

7.1.    Effectiveness    

7.1.1.    Surveys    

7.1.2.    Targeted interviews    

7.2.    Efficiency    

7.2.1.    Surveys    

7.2.2.    Targeted interviews    

7.3.    Coherence    

7.3.1.    Surveys    

7.3.2.    Targeted interviews    

7.4.    Relevance    

7.4.1.    Surveys    

7.4.2.    Targeted interviews    

7.5.    Added Value    

7.5.1.    Surveys    

7.5.2.    Targeted interviews    

Annex VI. Legislative background of Council Regulation (EC) No 515/97 and its role in the protection of the financial interests of the EU (Continuation of Chapter 2.1.1.4 above)

1.    Introduction    

2.    Objectives    

3.    Agricultural matters    

4.    Customs matters: scope and definitions    

5.    Assistance on request    

6.    Special watch on persons, goods and means of transport    

7.    Spontaneous assistance    

8.    Relations with the Commission    

9.    Transport directory    

10.    Container Status Messages (CSM)    

11.    Import, Export and Transit (IET)    

12.    Relations with third countries    

13.    Customs Information System (CIS)    

14.    Customs Files Identification Database (FIDE)    

15.    Data analysis    

Glossary

1.Introduction

This staff working document (SWD) on the interim evaluation of the Union anti-fraud programme (UAFP) 1 describes the achievements of the programme between early 2021 and early 2024, presents the findings, based primarily on an external study 2 , and suggestions as to the further implementation for the rest of the programme implementation period (2024-2027).

The UAFP, the successor to the Hercule III programme (2014-2020) 3 , was established by the adoption of Regulation (EU) 2021/785 on 29 April 2021 (‘the UAFP Regulation’), which entered into force retroactively on 1 January 2021, following the adoption in December 2020 4 of the 2021-2027 7-year EU budget (multiannual financial framework or MFF).

The UAFP, managed by the European Commission (through the European Anti-Fraud Office, OLAF) provides financial support to Member States’ law enforcement services and to academic and non-profit organisations to help protect the EU’s financial interests, i.e. keeping the EU budget safe from irregularities, fraud or any other illegal activities.

The programme also finances the operational management and development of OLAF's IT tool IMS (Irregularity Management System, the second component of the programme). This tool is used by Member States, participating countries, candidate countries and other beneficiaries of EU funding to report irregularities, confirmed fraud or suspected fraud related to European funding under shared and indirect management.

The third component finances the activities tasked to the Commission under Regulation (EC) No 515/97 5  (‘Council Regulation 515/97’) by means of a technical IT support platform, managed by OLAF, including the development of the AFIS (Anti-Fraud Information System). Member States’ law enforcement services and customs services use this technology platform, which consists of a set of applications operated under a common information system managed by the Commission, to exchange information and data.

Council Regulation 515/97 provides that the Union is to support mutual assistance between the administrative authorities of the Member States and cooperation between the latter and the Commission, to ensure the correct application of the law on customs and agricultural matters. That support is provided to a number of operational activities as described in Annex I of the UAFP Regulation.

The UAFP is set to run for the MFF from the beginning of 2021 to the end of 2027.

1.1.Purpose of the evaluation

The purpose of this interim evaluation of the UAFP has three aspects.

First, the Commission has to comply with a legal obligation. Article 13(2) of the UAFP Regulation provides that:

‘The interim evaluation of the programme shall be performed once there is sufficient information available about the implementation of the programme, but no later than four years after the start of the implementation of the programme.’

Therefore, under Article 13(4) of the same Regulation, the Commission ‘shall communicate the conclusions of the evaluation, accompanied by its observations, to the European Parliament, the Council, the European Economic and Social Committee, the Committee of the Regions and to the European Court of Auditors’ by 31 December 2024 at the latest.

Second, this evaluation should provide an overview of how the programme has worked so far, its achievements as well as its shortcomings in order to identify areas on which efforts can focus to improve implementation for the remaining period.

The last and third aspect relates to providing a basis for a potential impact assessment of the future programme for the next programming period, starting in 2028.

This evaluation assesses the programme’s effectiveness (including the sustainability/durability of certain actions in the longer term), its efficiency, the coherence (both internal and external), the relevance of the programme’s aim and objectives, and the added value at EU level. As this assessment had to be conducted for each of the three components of the programme and given their different characteristics, this SWD has turned out lengthier than expected.

1.2.The scope of the evaluation

The scope of the interim evaluation has been limited since only some activities and projects, funded by the programme since 2021, have been fully implemented and finalised (i.e. paid) within the timeframe set for the evaluation (early 2021 to early 2024).

In order to comply with the deadline of 31 December 2024 for the adoption and transmission of this report from the Commission to the other EU institutions, the external study supporting the Commission with this interim evaluation mainly covers the period between 1 January 2021 and 31 January 2024.

The first projects granted through the programme effectively started from early to mid-2022, following the time needed for the administrative procedure to receive, select, evaluate and award those projects. As most of the activities granted have a duration of between 18 to 24 months, those contracts that started in 2022 ran (or will run), at least, until the end of 2023 or throughout 2024 and 2025. The impact of COVID-19 also meant that many of the grant contracts’ duration was extended.

For this reason, only the results and outcomes of a reduced number of finalised activities could be evaluated, addressing the main questions in relation to the evaluation criteria:

-the relevance of the programme: does it make a difference and does it provide funding beyond the level of support that Member States can provide themselves?

-the effectiveness of the programme’s activities: can we measure an effect or an impact?

-the efficient use of human and financial resources: what are the costs and benefits for users? is it (still) worth funding this programme?

-is there internal consistency (coherence) of the programme’s activities, as well as external coherence with other similar EU funding programmes active in the field of fighting fraud, corruption or any other illegal activities?

-what is the EU added value of the programme? I.e. value that is additional to what would otherwise have been created solely by Member States acting on their own.

The sustainability of certain measures over the longer term will be considered under the ‘effectiveness’ criterion (durability of equipment or tools beyond the contractual period).

1.3.The methodology 

The methodology applied to this evaluation is described in Annex II. This staff working document is mainly based on the external study by the contractor ICF S.A. Belgium (ICF) 6 , which has been based on recognised evaluation techniques, including cross-checking methods. The study report explains the rationale for the action at the time it was prepared and adopted, i.e. identifying the problems or the needs the EU was trying to address and its underlying causes, what the programme expected to achieve and how that achievement was to be assessed 7 .

In the inception phase of the external study, the scope was adapted in agreement between the Commission (OLAF) and ICF. The main amendments, compared to the initial proposal by ICF before the contract started, included an analysis of each of the three components of the programme, their budgets, objectives, themes and priorities and a short overview of the annual call(s) for applications and the grant data.

This evaluation has also drawn, where possible, on the following sources:

-the 2018 ex ante evaluation report in preparation of the UAFP Regulation 8 ;

-the explanatory memorandum and the preamble of the final text of the UAFP Regulation.

The evaluation includes:

-the points of comparison against which the intervention has been assessed;

-wherever possible, a reference to one or more of the 17 UN Sustainable Development Goals 9 (SDGs) that the intervention is considered to address or support;

-case studies: The purpose of the case studies was to conduct a thorough examination of the main themes of the evaluation. The introduction of Regulation 2021/785 has brought about notable themes and innovations regarding the primary focus and structure of the UAFP. As the programme is still being implemented and many actions are ongoing, the main emphasis of the case studies was to assess progress in achieving the strategic direction outlined for the programme, rather than evaluating longer-term impacts and outcomes.

By taking this mid-term perspective on their design, the case studies offer a comprehensive understanding of whether the programme is advancing towards its objectives, allowing for adjustments if necessary for the remainder of the programme.

The four case studies 10 covered the following topics:

·Case study 1: investigative capacity

·Case study 2: digitalisation, AI, data analytics

·Case study 3: transnational cooperation and exchange of information

·Case study 4: training.

The results of the case studies have been incorporated into this document 11 .

This evaluation complies with the principles outlined in the Commission’s Better Regulation Guidelines 12 , which were adopted in May 2015 13  and updated on 3 November 2021 14 ;

The study and this evaluation cover:

-geographically, as does the programme, all 27 Member States, including one non-EU country (third country) for which an association agreement to the programme has become applicable retroactively on 1 January 2023, the association agreement with Ukraine, signed on 20 March 2024 15 .

-Where applicable, data relating to (reporting by) the United Kingdom 16 has been included in this SWD where appropriate.

-all adopted Commission Implementing Decisions on the annual work programmes (AWPs) within the scope (2021-2024). The Decisions concern the annual financing of the programme adopted, including (as an Annex to the Decision) the AWP, which details the allocated budget, the eligible bodies and activities for funding in a particular year, and the criteria for measuring the potential impact of the programme.

2.What was the expected outcome of the intervention?

2.1.The programme and its rationale, its objectives, its components and their rationale

1.

2.

2.1.

1

2

2.1

2.1.1The programme and its rationale

Fraud and related illegal activities, such as corruption, money laundering and illicit trade, pose a serious problem to the EU and Member States as such activities are liable to undermine both the EU's economy and national economies.

This problem is all the more pressing now that the EU’s total spending has grown to an unprecedented amount of around EUR 1.8 trillion, as established by the EU’s 2021-2027 multi-annual budget, that is, if one includes the Recovery and Resilience Facility (RRF) 17 , created in 2021 in response to the financial and economic impact of the COVID-19 pandemic. This is a record level of funding, and there are inevitable risks as to whether all funds will reach their intended purpose.

Efficient and proper spending of the EU’s budget, also in Member States, is key to maintaining the public and taxpayers’ trust and to boosting the strength and added value of European integration. Conversely, a high prevalence of fraud can stifle economic growth and undermine the broader economic recovery and fair competition. Reinforcing joint defences against fraud is therefore crucial in strengthening the EU’s resilience.

Article 325 of the Treaty on the Functioning of the European Union (TFEU) provides for a shared obligation of the EU and Member States to protect the EU's financial interests. Past experience shows that the EU's financial interests are impacted by illegal activities, such as fraud and illicit trade, and by financial and other irregularities. Every year, the ‘Annual Report on the protection of the EU's financial interests’ 18  (‘PIF Report’) demonstrates that the level of such irregularities, including fraud, although fluctuating, requires continuous action on the EU's part.

In addition, the current MFF was drawn up against the background of significant changes in 2017 to the legislative and institutional framework for the protection of the EU's financial interests, in particular the creation of the European Public Prosecutor’s Office (EPPO) 19 and the implementation of Directive (EU) 2017/1371 of the European Parliament and of the Council of 5 July 2017 on the fight against fraud to the Union's financial interests by means of criminal law (the ‘PIF Directive’) 20 .

The EPPO was set up to conduct criminal investigations and prosecutions, until cases are disposed of, in respect of crimes affecting the EU’s financial interests. The crimes falling within the material competence of the EPPO are set out in the PIF Directive 21 .

In more recent years, at EU level, the anti-fraud landscape has undergone major changes with, for instance, the operational start of the EPPO on 1 June 2021 and the recent revision of Regulation N° 883/2013 22 (‘the OLAF Regulation’), strengthening the effectiveness of administrative investigations carried out by OLAF.

The UAFP’s predecessor, the Hercule III programme (2014-2020), also had the general objective of protecting the EU’s financial interests to strengthen the competitiveness of the economy and protect taxpayers' money. The specific objective was to prevent and combat fraud, corruption and any other illegal activities affecting the EU’s financial interests. Action to protect the financial interests covered the whole expenditure side of the EU budget.

On the revenue side, action focused on the traditional own resources of the EU. Article 33 TFEU provides for strengthening customs cooperation among Member States and between Member States and the Commission.

Implementation of the EU multi-annual budget has been accompanied by the following set of measures to help Member States prevent and fight fraud affecting the financial interests of the EU, supporting mutual assistance in customs and agriculture matters.

(a) The Hercule III programme (2014-2020), supporting activities against fraud, corruption and any other illegal activities affecting the financial interests of the EU, now replaced by the UAFP. The UAFP Regulation covers the funding and implementation of the Hercule component (technical assistance, training and procured activities), the Anti-Fraud Information System (AFIS) and the Irregularity Management System (IMS).

(b) The Anti-Fraud Information System (AFIS) is an operational activity consisting essentially of a set of customs IT applications, running under a common information system, managed by the Commission (OLAF). It was set up to perform the tasks of protecting the financial interests of the EU, as entrusted to the Commission under Council Regulation 515/97, including the AFIS (Anti-Fraud Information System) to support mutual assistance between the administrative authorities of the Member States and cooperation between the latter and the Commission, to ensure the correct application of the law on customs and agricultural matters;

and,

(c) The Irregularity Management System (IMS) is a secure electronic communication tool that facilitates Member States’ obligation to report detected irregularities, including suspected or confirmed fraud cases, and supports the management and analysis of these cases and their related data.

Although it is difficult to quantify these measures’ financial impact, they have contributed to recovering large amounts of diverted money back to the EU budget. For example, Hercule III funded the development of the Automated Monitoring Tool (AMT), an IT tool that identifies anomalies in trade flows. It was used in several customs operations and was instrumental in identifying large undervaluation fraud schemes in the import of textile and footwear from non-EU countries.

From a budget implementation perspective, the IMS is an operational activity. The Commission is tasked with the IMS activity under sectoral legislation. The IMS needs to be maintained and developed sufficiently to allow Member States to comply with their reporting obligations. It also provides the Commission with the necessary data to analyse the main achievements in detecting and reporting irregularities, including fraud, relating to the EU's budget in shared management and pre-accession areas.

The financial support provided to the IMS, the AFIS, from a budget implementation perspective, is also an anti-fraud operational activity. In particular, where activities developed under the AFIS string of the programme, aim at tackling the increasing sophistication of criminal groups and new challenges arising in the fight against fraud, including from technological developments, there is still a real need for strenghtening the mutual administrative assistance activities between customs authorities and cooperation with the Commission.

2.1.2The programme’s general and specific objectives – context

The conclusions of the final evaluation of the Hercule III programme 23 and the challenges posed by the increased EU expenditure brought some adjustments to the approach adopted previously. This was reflected in the Regulation setting up the UAFP in 2021.

In particular, the programme now focuses more on the expenditure side of the budget, taking into account the new forms of EU expenditure, introduced by the MFF and RRF Regulations. It also puts an emphasis on new trends in crime, including cybercrime, reflected in the priorities set out in the AWPs of the programme.

The programme is objective-driven: every action it supports has to refer to the objectives outlined in the Regulation and in the AWP.

Due to the parallel adoption of the Customs Control Equipment Instrument (CCEI) 24 , managed by DG TAXUD in the Commission, focusing on providing financial support to customs services, consideration is given to targeting UAFP support to purchasing types of equipment that are not covered by the instrument.

Within this context, Article 2 of the UAFP Regulation describes the general and specific objectives of the Union anti-fraud programme, as presented in the table hereafter.

Table 1: The general and specific programme objectives

The main beneficiaries of the programme are the Member States’ authorities involved in the fight against fraud although the programme also provides for the participation of non-EU countries (Ukraine, e.g.) under certain conditions as explained in more detail in Chapter 3.2.4.6.

2.1.2.1The Hercule programmes (I-II-III)

In 2004, the Hercule programme was set up by Decision No 804/2004/EC of the European Parliament and of the Council. It was amended and extended by Decision No 878/2007/EC of the European Parliament and of the Council (the ‘Hercule II programme’). This has made it possible for the past 20 years to promote and improve the activities carried out by the EU and Member States to counter fraud, corruption and any other illegal activities affecting the EU's financial interests.

Although tangible progress has been achieved over these last two decades through the first Hercule programme (2004-2007), the Hercule II programme (2007-2013) and the Hercule III programme (2014-2020), the level of protection of the EU’s financial interests among Member States remains uneven.

This differentiation can be influenced by various factors. These include: (i) different enforcement and/or government priorities; (ii) different technical capacities (the UAFP might not be the only programme/funding tool the Member State is working with); (iii) a different level of PIF awareness in the Member State; (iv) the geographical location in Europe (especially on the revenue side); (v) the size and wealth of the Member State in question and the related resources they allocate or wish to allocate to law enforcement and justice administrations; and (vi) countries' differing levels of fraud and corruption.

The Hercule III programme ended on 31 December 2020. The programme provided protection by financially supporting national and regional administrations, research and educational institutes and other non-profit making entities, which ‘promote the strengthening of action at Union level to protect the financial interests of the Union’, including comparative research and scientific publications on fraud-related topics.

The total budget for Hercule III amounted to EUR 104.9 million over 2014-2020 (UAFP 2021-2027: EUR 181.2 million or 72.7% more). The programme was implemented on the basis of annual cycles that carried out the AWPs.

The 2018 mid-term evaluation of the Hercule III programme found that, like the final report of the Hercule II programme, stakeholders almost unanimously recognised the added value of the programme and its essential contribution to generating benefits 25 . More specifically, the programme’s added value was principally seen as acting as an enabler for cross-border cooperation, cross-border exchanges of information and the sharing of best practices, including on cybercrime.

The Commission submitted in December 2021 the ‘Report from the Commission to the European Parliament and to the Council’ 26 , with an accompanying staff working document, on the final evaluation of the Hercule III programme and its merits with regard to relevance, efficiency, effectiveness, coherence, added value and durability (sustainability).

This Commission report was supported by a study by an external contractor (Ramboll (Belgium) S.A.), providing the necessary elements to submit conclusions and recommendations to the legislators 27 .

The conclusions of both the mid-term and final evaluations of the Hercule III programme found that the programme was successful in meeting its objectives and providing EU added value, contributing to the general objective of the programme.

Hercule III was seen as highly relevant in addressing the main problems faced by stakeholders because it complemented the work carried out by Member States. The stakeholders consulted indicated a high level of satisfaction with the programme.

The final evaluation of the Hercule III programme 28 confirmed the success of the programme and concluded that the programme had been effective. The technical assistance measures (such as financing the purchase of scanners to detect smuggled goods or funding training in digital forensics) supported the law enforcement authorities in Member States in their work to combat illegal cross-border activities.

However, the evaluation also found aspects that could hamper the effectiveness of the programme. These included the lack of resources in Member States’ administrations (which is not an aspect the Commission could influence or change) and the corresponding lower ability to cope with the administrative requirements, which could have led to fewer applications.

The final evaluation showed that the programme had been efficient across both training activities and technical assistance projects. The benefits of the programme were found to outweigh the costs incurred by the beneficiaries.

Differences in the administrative capacity of Member State administrations posed a challenge that could be overcome by providing more guidance for applicants and, where possible, simplifying the procedures.

In terms of internal and external coherence, no specific overlaps or duplication were identified. The evaluation identified a high degree of complementarity between the Hercule III programme and the AFIS.

The overall view of all stakeholders was that the programme provided clear EU added value. The programme acted as an enabler and driver for Member States to use a higher standard of equipment to detect and reduce the risks of fraud than would have been possible without the programme. The programme was also seen to act as a facilitator for EU-level cooperation between Member States, specifically through the funding of training activities, conferences, funded access to databases and associated tools.

70% of the 99 respondents consulted at the time by the contractor Ramboll stated that the same results could not have been achieved at national or regional levels. Crucially, the evaluation showed that a withdrawal of the programme would have led to an increase in fragmentation between Member States and created an uneven playing field in the fight against fraud, resulting in uneven protection across the EU.

Moreover, respondents indicated that the Hercule III programme served as a deterrent. Therefore, withdrawing the programme risked sending a message that fighting fraud is not a priority or no longer a priority, thereby potentially emboldening criminal actors 29 .

2.1.2.2The Union anti-fraud programme (2021-2027)

Acknowledging the contribution of the Hercule III programme and based on the 2018 ex ante evaluation of the UAFP Regulation 30 , the Commission proposed a new Regulation, establishing the UAFP, to start under the 2021-2027 MFF.

The legislative proposal was adopted on 29 April 2021 as Regulation (EU) 2021/785, with a total budget of EUR 181 207 000. The programme has been established for 7 years to align its duration with that of the MFF laid down in Council Regulation (EU, Euratom) 2020/2093 31 .

The 2018 ex ante evaluation of Hercule III highlighted the following needs to which the proposed programme should respond (summarised):

-the lack of up-to-date technical equipment and support in Member States;

-the rise in new forms of fraud and the rapid development of sophisticated technical tools used by criminals 32 ; 

-more cooperation between stakeholders (including the exchange of best practices and experiences);

-access and exchange of data and information between stakeholders and Member States.

The independent study underpinning the Commission ex ante evaluation outlined some ideas that have been proposed, including possible objectives and activities for the programme going forward. This includes cross-border cooperation between Member States and cooperation with non-EU partners, including on technological challenges.

The study concluded, on the basis of the final reports of the actions and surveys of beneficiaries, that training activities generated results that were largely in line with the expected output. Nonetheless, it found that only a few training activities involved staff exchanges between national administrations or international participation.

Most of the suggestions from the 2018 evaluation have been addressed, as the Commission, since 2017, has been using an electronic management system for the submission, processing and management of grant applications under the Hercule III programme. The improvements brought about by the electronic management system include digitalising the whole application process and introducing the use of digital signatures. The electronic system has noticeably alleviated the administrative burden and the related time and resources spent by applicants on their proposal, as well as facilitated the monitoring of earlier and ongoing or closed projects.

The programme’s intervention logic was drafted, based on the identified needs, the established objectives, the known financial input, the planned activities and the expected results and impact.

The UAFP is the main EU-level programme specifically designed to protect the EU’s financial interests. The UAFP may act in synergy with other EU instruments providing technical support in the fight against fraud and corruption, such as the Technical Support Instrument (TSI) 33 , as well as other EU programmes, such as the Customs programme 34 , the CCEI and the Fiscalis programme 35 .

Although the IMS is an IT tool running under the platform provided by the AFIS, the AFIS is considered to be the third component of the programme (and not the second). This is because of how the budget line numbering was attributed to these components during the legislative preparatory activities. During the adoption process of the UAFP Regulation, the same order was maintained 36 . This meant that the IMS was mentioned as the second component (although financially the smallest) before the third component, the AFIS (in light green below).

Figure 1: Intervention logic of the UAFP 

As of July 2024, 24 of the 27 Member States 37  participate in the EPPO 38 . Therefore, different forms of cooperation between the EPPO, the European Commission and national judicial and administrative authorities have been established. Specifically, for the purpose of establishing and maintaining a cooperative relationship, the EPPO and the Commission have concluded an agreement on 18 June 2021 39 while the EPPO and OLAF concluded a working arrangement on 5 July 2021 40 . 

A major change in the recent implementation of the UAFP has been the adoption of a Commission Decision 41  allowing Ukraine to become an associated candidate country to the programme. See Chapter 3.3 for details.

Table 2: Overview of the UAFP characteristics

2.1.2.3Aim and rationale of the UAFP’s Hercule component (protection of the EU’s financial interests)

As mentioned earlier, in response to the COVID-19 pandemic crisis, the EU adopted the 2021-2027 MFF and the RRF 42 . With a total amount of EUR 1.8 trillion, this is the largest package ever financed through the EU budget. It is therefore more important than ever to pursue the objective of protecting this budget, which represents EU taxpayers’ money, and to ensure that it is collected and spent correctly.

For the expenditure side of the EU budget, the UAFP aims to help Member States implement various types of budget management, whether direct, indirect or shared.

To achieve this objective of protecting the EU budget as best as possible, together with the Member States, the EU has recently strengthened its anti-fraud architecture, featuring:

-a European Anti-Fraud Office (OLAF) with strengthened investigative powers through a revision of the OLAF Regulation in 2020;

-the criminal investigative and prosecutorial powers of the EPPO;

-the analytical capacity of the EU Agency for Law Enforcement Cooperation (Europol);

-the coordinating role of the EU Agency for Criminal Justice Cooperation (Eurojust).

The UAFP complements this strengthened regulatory framework by offering direct and indirect financial support to Member States, bearing a joint responsibility to counter fraud, corruption and any other illegal activities affecting the EU’s financial interests.

To this effect, this first component of the UAFP allocates EUR 114 207 000 for 2021-2027, of which EUR 16 075 789 has been allocated for 2024. This budget funds activities to fight fraud, corruption and any other illegal activities affecting the EU budget.

The Commission implements this first component of the UAFP by:

-awarding grants following annual calls for proposals (direct support);

-concluding public procurement contracts following calls for tender (indirect support);

-entering into administrative arrangements with the Commission’s Joint Research Centre (JRC), providing indirect support;

-reimbursing costs incurred by representatives from eligible entities who participate in training and operational activities (direct support).

To be eligible for funding, proposed actions must implement the UAFP’s objectives. Financial resources allocated to the UAFP are also intended to contribute to the corporate communication of the EU’s political priorities, in particular those related to the UAFP’s objectives.

Some of the actions financed under Hercule, this first component of the UAFP, are therefore also likely to help deliver on the EU’s current political priorities by:

-preparing for the challenges and opportunities of the green and digital transitions;

-making EU economies and societies more resilient to shocks in the future, provided these actions are linked to the UAFP’s objectives.

It is important to tackle climate change in line with:

-the commitments of the EU to implement the Paris Agreement adopted under the UN Framework Convention on Climate Change; 

-the UN Sustainable Development Goals 43 .

Reflecting the importance of these efforts, the UAFP aims to help ensure that climate action is integrated into all policy areas and aims at helping achieve an overall target of 30% of the EU budget supporting climate objectives.

Therefore, the UAFP may, for example, enable Member States to be better equipped to prevent the import of certain illicit products that do not comply with standards laid down in EU environmental and climate legislation. In doing so, the actions taken under the UAFP not only prevent infringements of EU customs legislation but also protect the environment (as one example of the EU added value of the programme).

The UAFP can also support cooperation between customs and other competent authorities to fight certain fraudulent activities that damage the environment and people’s health, such as activities related to illegal waste shipments, the illicit trade in endangered species, illegal chemical products and fraudulent or substandard food products.

The UAFP also strives to support Member States’ digital transition, for instance, by:

-funding the purchase of IT tools supporting data analysis to detect fraud that affects revenues or expenditures;

-continuing to build national expertise on digital forensics (through specialised training).

2.1.2.4Aim and rationale for the Anti-Fraud Information System (AFIS) platform

The main legal basis for the operation of the AFIS is Council Regulation 515/97.

The programme supports mutual administrative assistance activities under Council Regulation 515/97, including the AFIS. This assistance helps ensure a correct application of the law on customs and agricultural matters.

For the legislative background of Council Regulation 515/97 and its crucial role in the protection of the financial interests of the EU 44 , see Annex VI for a more detailed description.

Council Regulation 515/97, together with the UAFP Regulation, are the cornerstones of mutual assistance in customs and agricultural matters at EU level. Effective cooperation in these fields strengthens the protection of the EU’s financial interests and contributes to the safety and health of citizens and the protection of the environment.

AFIS is an IT platform that consists of a set of anti-fraud IT applications, run by OLAF under a common technical infrastructure. The aim of this set of IT applications is the timely and secure exchange of fraud-related information between national authorities and EU institutions, bodies, offices and agencies, as well as the storage and analysis of relevant data.

The AFIS platform has more than 9 000 registered end users in Member States, partner third countries, international organisations, the Commission and other EU institutions. AFIS enables substantial economies of scale and synergies in developing, maintaining and operating such a wide and diverse set of IT services and tools.

AFIS supports mutual assistance in customs by providing collaboration tools such as:

-VOCU (Virtual Operations Coordination Unit) for joint customs operations;

-secure web mail (AFIS Mail);

-databases such as CIS (Customs Information System) and FIDE (Customs Investigation Files Identification Database);

-the Container Status Messages (CSM) directory;

-the Import, Export and Transit (IET) directory.

AFIS also provides support in the form of data analysis tools, such as AMT (Automated Monitoring Tool) and a platform for strategic and operational data analysis. AFIS also supports electronic workflow applications such as ToSMA (the Tobacco Seizures Management Application).

This third component of the UAFP also covers expenditure relating to the use of the CIS database, provided for in instruments adopted under Article 87 TFEU 45 (police and law enforcement agencies’ cooperation in combating crime), in particular in Council Decision 2009/917/JHA 46 , insofar as those instruments require such expenditure to be borne by the EU general budget.

Through this component, EUR 60 000 000 has been allocated through the UAFP for 2021-2027, of which EUR 8 445 091 has been reserved for 2024, to provide tools for information exchange and support for operational activities.

2.1.2.5Aim and rationale for the IMS tool

The second component of the UAFP finances the development and maintenance of the IMS, which supports the reporting of detected cases of irregularities and fraud.

The IMS is a secure electronic communications tool, operated under the AFIS platform, which helps Member States, candidate countries and potential candidate countries fulfil their obligation to report detected irregularities. The IMS also enables managing and analysing the reported data, which is a unique tool within the community of law enforcement agencies and of the management and control systems of EU funds.

For this component, EUR 7 000 000 has been allocated through the UAFP for 2021-2027, of which EUR 985 119 has been allocated for 2024.

EU legislation for various funds requires Member States, candidate countries, potential candidates and third countries to report irregularities and fraud affecting the EU’s financial interests. This obligation can be found in the EU legislation that sets out rules for:

-the European Agricultural Guarantee Fund;

-the European Agricultural Fund for Rural Development;

-the European Regional Development Fund;

-the European Social Fund;

-the Cohesion Fund;

-the European Maritime and Fisheries Fund;

-the Asylum, Migration and Integration Fund;

-the Instrument for financial support for police cooperation, preventing and combating crime and crisis management;

-the Fund for European Aid to the Most Deprived;

-the Fund for pre-accession assistance;

-the Recovery and Resilience Facility (here exclusively on a voluntary basis).

2.1.2.6New features in the UAFP programme since 2021

The UAFP introduced major changes from the now concluded Hercule III programme, as clarified by the Regulation. One of these changes is the circle of entities eligible for support. Under Article 10 of the UAFP Regulation, the eligibility of non-EU countries now generally depends on the conclusion of association agreements in accordance with Article 4, binding these countries to the implementation of the programme. This was the case for Ukraine, which was associated to the programme in February 2024, with retroactive applicability from 1 January 2023.

Another new feature is set out in Article 10(2)(c) of the UAFP Regulation, which opens the possibility for international organisations to participate in the programme.

The addition of any international organisation has been inspired, during the preparatory legislative work on the Regulation, by the fact that the programme was initially supposed to directly fund the annual contribution for the running costs of the secretariat of the Framework Convention for Tobacco Control (‘FCTC’). This particular funding required the possibility for specific indirect budget management in the programme.

Therefore, Article 5(1) of the UAFP Regulation allowed the possibility of indirect management implementation, although the main components of the programme are implemented through direct management. Funding the annual contribution to the FCTC, through a specific budget line, is done with a specific Commission implementing decision on financing, including a work programme for the FCTC, covering each time a period of two years.

Another change, impacting the practical focus of the programme’s support, arises from the parallel adoption of Regulation (EU) 2021/1077 , establishing a Customs Control Equipment Instrument (CCEI), as part of the Integrated Border Management Fund. Both are managed by DG TAXUD, the Commission department dealing with taxation and the customs union.

CCEI is equipped with a substantial budget of around EUR 1 billion for the current budget period. It can offer support in an area that had received substantial support under the past Hercule programmes. The Commission pointed out that technical assistance under the UAFP will now be targeted at the purchase of types of equipment that are not covered by the CCEI in order to avoid any overlap or duplication of EU financial support.

As a result, implementation of the programme is now focusing more on expenditure fraud. One of the recent developments this interim evaluation has looked into is the use of UAFP funding to build up national capacity to better protect expenditure against fraud.

In a shift away from costly customs equipment, the UAFP should also free up resources to strengthen support for other state-of-the-art equipment for operational anti-fraud work, such as advanced data analytics technologies and data mining tools. This more data-oriented approach is also in line with the 2019 Commission’s anti-fraud strategy (CAFS), which highlights improved data analysis as a tool to better detect fraud.

1.1.1.1.

2.2.Points of comparison 

2.1

2.2

2.2

2.2.1Situation before the start of the intervention

This section assesses the baseline scenario of what would have happened if the situation before the adoption of the UAFP had remained the same compared with the results and progress achieved to date.

As set out in the Better Regulation Toolbox 47 (Tool #60), a baseline is a ‘no-policy-change’ 48   scenario that makes it possible to assess how the UAFP has evolved since the last programming period. The baseline gives the evaluation a comparison point and benchmark against which to assess the UAFP achievements in meeting its objectives.

The points of comparison that could be identified in the legislative financial statement, i.e. the annex to the ‘Explanatory memorandum’, which is part of the May 2018 ‘Proposal of the Commission for a Regulation establishing an EU anti-fraud programme’ 49 , are as follows:

-Added value of EU involvement (For the purposes of this point 'added value of EU involvement' is the value resulting from EU intervention that is additional to the value that would have been otherwise created by Member States alone.)

oReasons for action at European level in 2018 50 :

Fraud affecting the EU's financial interests is a cross-border phenomenon, affecting all EU Member States, as shown by statistics published in ‘the PIF Reports. There is a need for coordinated action to tackle fraud and also to facilitate mutual administrative assistance in customs matters. Hercule, AFIS and the IMS are all well-established tools supporting the Member States and the EU in this respect. This proposal aims to streamline the financial support for these tools for the duration of the next MFF.

oExpected generated EU added value:

The programme is expected to continue supporting the Member States and the EU in fighting fraud affecting the EU's financial interests by financing activities (technical assistance and training), which otherwise might not be available at national level in all Member States. It will also provide services (AFIS and IMS) that, to meet their objectives, should be delivered in a centralised and horizontal manner for all Member States. The programme will also allow for savings as has been the case with the centralised purchase of access to various databases.

-Lessons learnt from similar experiences in the past (before 2018):

Hercule, AFIS and the IMS are well-established EU tools for combating fraud and irregularities that harm the EU budget and for supporting mutual administrative assistance between customs administrations and cooperation with the Commission. They have been subject to regular evaluations and user satisfaction surveys, which have largely demonstrated their added value.

As an operational activity, AFIS has been subject to user satisfaction surveys, internal audit activities, as well as participants’ evaluations, joint customs operations (JCOs) and training sessions. AFIS was positively appreciated, showing a satisfaction rate of more than 80% among users for functionality, performance and training activities. Moreover, the European Court of Auditors concluded that the information on Container Status Messages and fair prices, provided by the AFIS applications CSM directory and Automated Monitoring Tool, are effective tools in identifying potential cases of fraud related to the misdescription of origin and undervaluation/underinvoicing.

The IMS is analysed in close collaboration with Member States, as part of the ‘Reporting and Analysis’ component of the COCOLAF 51 advisory committee of the Commission (OLAF).

-Compatibility and possible coordination with other appropriate instruments:

The conclusions of the final evaluation of the Hercule III programme and the new challenges posed by the increased EU expenditure brought some changes to the new programme (2021).

Most notably, the programme now focuses more on the expenditure side of the budget, taking into account the new forms of EU expenditure introduced by the MFF and the RRF Regulation. It also puts a stronger emphasis on new trends in crime patterns through its AWPs, reflecting possibilities for funding projects related to a non-exhaustive list of new or developing trends. Continuous coordination between Commission services managing programmes that are related to the objectives of the UAFP, such as the CCEI or the Customs programme, ensures that potential financial double funding is avoided. This coordination takes place, in particular, during the annual evaluation exercise of new applications received for funding by these programmes or instruments.

Furthermore, the ex ante evaluation of the UAFP demonstrated the clear EU added value of AFIS and IMS. The EU added value of AFIS is linked to the critical importance of the effective performance of the national customs authorities’ missions, in collaboration with the Commission. The contribution of AFIS to cross-border coordination, especially in terms of collection of and access to data, proved vital to this effective performance.

IMS is the only database in the EU that consolidates reported information about irregularities, including fraud, in the shared management and pre-accession funds. Information is available by field of EU action and by Member State (and reporting country). It allows for various types of analyses and helps further develop an evidence-based EU anti-fraud policy. Through the analyses developed by OLAF, it also feeds into the Member States’ fraud risk assessments.

The added value of such an EU-wide framework allows for identifying common risks and patterns and drawing up and implementing common approaches to fighting fraud. This contributes to the effective and equal protection of the EU’s financial interests throughout the EU.

The following overview table presents the indicators to measure the performance of the UAFP and how its components performed in the context of these indicators. They are listed per evaluation criterion and have been developed by the contractor ICF and further refined through desk research, literature review and ICF’s consultations with stakeholders.

For the development of the baseline, particular attention was placed on the Commission’s PIF Reports, the final evaluation of the Hercule III programme and the 2019 AFIS Survey summary report 52 .

2.2.2Overview of the points of comparison by evaluation criterion

Table 3: Points of comparison identified per evaluation criterion

Source: ICF analysis

1.

2.

2.1.

2.2.

2.2.1.

1.

2.

2.1.

2.2.

1.

2.

2.1.

2.2.

2.2.1.

2.2.2.

3.How has the situation evolved over the evaluation period?

3.1.Implementation of the programme

The UAFP is being implemented by the Commission through the adoption of AWPs, which cover each year of implementation.

The Commission adopts these work programmes under Article 11 of the UAFP Regulation and Article 110 of Regulation (EU, Euratom) 2018/1046 57 , (the ‘Financial Regulation’). On that basis, each work programme sets out the priorities for funding in a particular year, as well as the objectives pursued, the expected results (using indicators) and the methods of implementation. The work programme also responds to specific requirements and needs arising from the UAFP Regulation’s provisions and the feedback received from beneficiaries and stakeholders (internal and external) during exchanges of information, evaluation reporting, workshops, etc.

During implementation of the programme, OLAF measures the fulfilment of the specific objectives through two performance indicators upon finalisation of the project(s) or action(s).

The first indicator relates to the added value and effective use of the co-financed technical equipment funded under the UAFP (as reported by direct users). In the final project reports for 2021, beneficiaries expressed a 97% satisfaction rate for the equipment funded under the programme, surpassing the target of 75% 58 .

In addition, the programme performed well in the second performance indicator, which measures the number of training activities funded and the associated satisfaction rate. For the training grants finalised in 2021, participants and trainees had a satisfaction rate of 91% and considered that the activities were very well-suited to their needs 59 .

Example: The 2024 AWP: Indicators to measure the expected results (excerpt)

‘The Commission will support the following four categories of technical assistance actions. The expected results from the actions, together with the indicators enabling them and the added value and effective use of the co-financed technical equipment to be measured, are as follows:

1.investigation and surveillance equipment and methods:

The purchase and maintenance of investigation and surveillance equipment and methods used by beneficiaries in the fight against irregularities, fraud and corruption, detrimental to the Union’s financial interests. The purchase of adapted transport equipment, IT hardware and software and audio-visual equipment may be included, provided an applicant clearly demonstrates that the purchase helps to achieve the first specific objective of the UAFP. Specialised training to enable staff to operate these tools has to be planned as part of the action.

2.digital forensic hardware:

The purchase and maintenance of digital forensic equipment and software, mobile forensic tools and computer forensic collaborative systems used in the fight against (fraudulent) irregularities, fraud and corruption detrimental to the Union’s financial interests. Cross-border cooperation enabling the exchange of information and best practices, in particular at operational level, is strongly encouraged.

3.data analytics technologies and data purchases:

The purchase and maintenance of commercial specialised databases, data analysis platforms capable of running analysis in big data environments, risk and predictive analysis and data mining tools, as well as systems supported by artificial intelligence used in the fight against irregularities, fraudulent activities and corruption, detrimental to the Union’s financial interests. These purchases include capacity building in Member States for developing, using and sharing databases and business intelligence tools. These purchases may also enable the acquisition of integrated packages, comprising for example hardware and software, access and training, including training for the ‘Tobacco Tracking & Tracing’ database.

Expected results for actions 1-3: Strengthening and improvement of beneficiaries’ investigative and operational capacity, as measured, for example, by the number of successful operations carried out with the purchased products in support of investigations into activities detrimental to the Union’s financial interests. This includes the number of arrests, convictions, seizures, confiscations, recoveries, prevented losses to the national and Union budgets, and fraud schemes uncovered.

4.detection of illicit trade:

The purchase and maintenance of equipment to strengthen beneficiaries’ operational and technical capacity to detect smuggled and counterfeited goods, including cigarettes and tobacco, imported into the Union with the intention of evading VAT, customs duties and/or excise taxes.

Expected result: Strengthening and improvement of beneficiaries’ (in particular, customs authorities’) technical capacity to carry out verifications of trucks, containers and (other) vehicles, as measured by the number of verifications and ‘hits’ following the use of the equipment.’

In order to avoid any duplication in EU support, the UAFP targets its support at the acquisition of types of equipment that do not fall under the scope of the CCEI or for equipment for which the beneficiaries are authorities other than the authorities targeted by that instrument. The Commission consistently considers any potential overlap while evaluating the coherence and added value of a proposal for funding by the programme.

Figure 2: Intervention logic of the UAFP, established by ICF  60

3

3.1

3.1.1

3.1.2

3.1.3

3.2The funding of the programme’s components and actions

3.2.1Budget and budget implementation

Article 3 of the UAFP Regulation has set the available UAFP budget for the 2021-2027 MFF at EUR 181.2 million in current prices.

According to the UAFP Regulation, the programme will be implemented, in principle, in direct management in line with the Financial Regulation, or (rather exceptionally) in indirect management with a body (for example, an international organisation) referred to in Article 62(1), first subparagraph, point (c) of the Financial Regulation.

Under Article 5 of the UAFP Regulation:

-The programme may provide funding in any of the forms laid down in the Financial Regulation, in particular grants and procurement, as well as the reimbursement of travel and subsistence expenses as provided for in Article 238 of the Financial Regulation. The programme may provide funding for actions carried out in line with Council Regulation (EC) No 515/97, in particular, to cover the types of costs referred to in the indicative list in Annex I to that Regulation.

-Where the supported action involves the purchase of equipment, the Commission is required, if appropriate, to set up a coordination mechanism to ensure efficiency and interoperability between all the equipment purchased with the support of other EU programmes (e.g. through DG TAXUD’s CCEI).

Grants may cover up to 80% of the eligible costs. In exceptional and duly justified cases, if the proposed project is a ‘priority action’, this percentage may be increased to a maximum of 90% 61 under the conditions set out below 62 .

1)For the technical assistance call, the percentage may be increased when:

(i) the proposal reflects the findings of the PIF reports, in particular by identifying situations that are vulnerable and that pose the greatest threat to protecting the EU’s financial interests; or

(ii) the proposal reflects the findings of the European Court of Auditors’ Special report 19/2017 on import procedures 63 .

2)For the training call, the percentage may be increased when:

(i) projects undertaken by scientific and/or research organisations fulfil at least one of the criteria (a) and (b) below, in addition to criterion (c):

(a) the project has been specifically created to promote studies in European criminal law; or

(b) the project supports the creation of networks in this area; and

(c) these activities are geared towards protecting the EU’s financial interests; or

(ii) training projects cover one of the exceptional cases listed for the technical assistance call.

Public procurement contracts may be awarded to any legal entity that satisfies the requirements set out in the applicable legal framework. Public procurement is used for purchasing (access to) databases, conferences and (specialised) training and the development of specific IT tools.

3.2.2Annual work programmes (AWP)

In the Annex to the annual Financing Decisions, there is an overview of the activities to be carried out in a given year and the available budget per sector of activities (technical assistance and IT support, training, conferences, access to databases, etc.). It also lists the calls for proposals to be published as well as the contracts to be concluded with the budget of that year.

The table presented hereafter provides detailed figures on the adoption dates, references and amounts allocated to the different components of the programme, starting with the year 2020, i.e. the last year of implementation of the Hercule III programme.

Table 4: Overview table on adopted acts (Financing Decision and annual work programmes / annual overviews), situation as of 2 July 2024

3.2.3Monitoring and reporting method

Article 12 of the Regulation states that:

‘Indicators to report on the progress of the programme towards the achievement of the general and specific objectives laid down in Article 2 are set out in Annex II [to the Regulation].’

To ensure an effective assessment of the programme’s progress towards the achievement of its objectives, the Commission is empowered to adopt delegated acts, in line with Article 14, to amend Annex II with regard to the indicators where considered necessary, as well as to supplement this Regulation with provisions on the establishment of a monitoring and evaluation framework.

The Regulation sets out that the Commission should report or continue to report annually on the performance of the programme to the European Parliament and to the Council in the framework of its ‘Annual Report on the protection of the Union’s financial interests – Fight against fraud’. The progress of the implementation of the programme is measured through an ‘Annual Overview’, annexed to the PIF Report (See Table 4 above).

As part of the discussions on the PIF reports, the European Parliament may make recommendations for the annual work programme. The Commission must duly take those recommendations into account.

The Commission’s annual performance reporting system must ensure that data for monitoring the implementation and the results of the programme are collected efficiently, effectively and in a timely manner. To that end, proportionate reporting requirements must be imposed on recipients of EU funds and, where relevant, on the Member States. 67

In line with Article 12(1) of the UAFP Regulation and its Annex II, the programme implementation is monitored through a set of key indicators, in relation to which data will and are being collected, per objective, per year of implementation, published year +1 in the ‘Programme Performance Statement’ by the Commission, per programme 68 .

-    Specific objective 1 (‘Hercule’): Preventing and combating fraud, corruption and any other illegal activities affecting the financial interests of the EU.

•    Indicator 1: Support in preventing and combating fraud, corruption and any other illegal activities affecting the financial interests of the EU, as measured by:

§1.1: the satisfaction rate of activities organised and financed or co-financed through the programme;

§1.2: the percentage of Member States receiving support each year from the programme.

-    Specific objective 2 (‘IMS’): Supporting the reporting of irregularities, including fraud, with regard to the shared management funds and pre-accession assistance funds of the EU budget.

•    Indicator 2: the user satisfaction rate for the use of IMS.

-    Specific objective 3 (‘AFIS’): Providing tools for information exchange and support for operational activities in the field of mutual administrative assistance in customs and agricultural matters.

•    Indicator 3: the number of instances in which mutual assistance information is made available and the number of supported mutual assistance-related activities.

The programme has a limited financial size and is managed by a small team in OLAF. In addition, the initial intention of the legislators was not to create a large set of performance indicators in the UAFP Regulation so that the administrative burden would stay reasonable. For these reasons, OLAF applies an internal working method on how it monitors and evaluates the implementation and outcome of the programme’s actions. This includes selected on-the-spot visits to beneficiaries’ premises to verify the existence of the funded purchases for technical assistance (equipment).

This method also takes into consideration the result indicators mentioned above for each specific objective, as well as certain input and output indicators, in particular on budget and programme implementation (consumption of the budget, number of grant agreements and contracts, etc.).

Consideration has to be made for the fact that COVID-19 caused the duration of a considerable number of projects to be extended because certain equipment was not delivered in 2021 and 2022. This resulted in final payments being postponed until the delayed final report was received. For this reason, payments were also delayed, showing likely a delayed ‘consumption’ of the annual budget.

Furthermore, the Commission (OLAF) also monitors the impact of the programme through information in grant beneficiaries’ or contractors’ final technical reports. These reports cover the results achieved with the activities funded under the programme and the contribution made to the protection of the EU’s financial interests. This reporting accompanies their request to OLAF for final payment.

For technical assistance, this information consists, for example, of the number of seizures of cigarettes or counterfeit goods that were made with the purchased technical equipment. Beneficiaries also report (where available or quantifiable) on the estimated financial impact of seizures and the losses to national and EU budgets as well as the number of persons under investigation or detained suspects in relation to seizures or other operations.

Moreover, beneficiaries of technical assistance grants must submit a final implementation report within 1 year after the closing date of a grant agreement. Through this final implementation report, the beneficiary can report more extensively on the achievements linked to the purchased equipment and demonstrate the contribution of the action and its results in achieving the programme’s objectives.

The beneficiaries are requested to indicate what difficulties they had to overcome during the implementation of the grant agreement, such as procurement problems or difficulties with the installation or maintenance of the technical equipment.

For conferences, seminars and training events, the final technical reports contain the results of participant surveys on the usefulness and relevance of the activities for daily work and/or the fight against fraud.

3.2.4Actions under the Hercule component – analysis

Table 5: Overview of the Hercule budget between 2014 and 2020

The allocated budget has increased significantly over time with each new programming period, i.e. from EUR 12 million for Hercule I (EUR 4 million per year) to EUR 98.5 million for Hercule II and EUR 104.9 million for Hercule III. The UAFP has a budget of around EUR 181.2 million for 2021-2027 69 of which the majority, 63%, is allocated to the Hercule component (EUR 114.207 million).

The second largest part, 33%, is allocated to the AFIS component and the remaining part, 3.8%, is allocated to the IMS component.

3.2.4.1Type of calls

Between 2021 and 2023, the Hercule component of the programme awarded grants to 98 projects (74 for technical assistance and 24 for training and studies).

Figure 3: Hercule component by type of calls (2021-2023)

3.2.4.2Activities and output

The awarded projects included a variety of procurement (purchase of equipment) and training activities, reflecting a wide range of needs. These needs included acquiring new systems and equipment to boost investigative capacity and capabilities, upgrading systems and training staff to take advantage of technological advancements (e.g. digital forensics, deployment of artificial intelligence), and sharing information and best practices among competent authorities in charge of combating fraud and irregularities.

Based on the review of the calls for proposals and the project applications, the planned activities and output were grouped into the categories listed below.

3.2.4.3Technical Assistance

1.investigation and surveillance equipment and methods

2.digital forensic hardware

3.data analytics technologies and data purchases

4.detection of illicit trade.

3.2.4.4 Training

1.specialised training sessions

2.conferences, workshops, and seminars

3.staff exchanges

4.comparative law studies and periodical publications. 

Figure 4 below presents the relative weight of the different types of activities/output.

Figure 4: Hercule component by type of activities and output (2021-2023 calls)

Source: ICF analysis of the documentation of the Hercule component projects

3.2.4.5Size of grants

The size of grants varies from over EUR 1 million for three technical assistance projects to less than EUR 50 000 for some training grants (minimum threshold: EUR 40 000 for training). About half the overall grant amount for the Hercule component was awarded to 20 projects with grants over EUR 500 000. A total of 29 technical assistance projects have grants below EUR 300 000, amounting to 17% of the total grant amount.

Most grants for training activities and conferences are around or below EUR 100 000, with only three grants above EUR 200 000 (one for a publication and two for comparative studies).

3.2.4.6Beneficiary countries

Between 2021 and 2023, grants were awarded to applicants from 17 Member States.

After an evaluation of all the received applications, over 77% of the total grant amount and almost 80% of the number of projects were awarded to beneficiaries from six Member States: Romania, Italy, Latvia, Spain, Poland and Lithuania.

The uneven distribution of grants is mostly due to the lack of applications from many of the eligible countries (and to the differences in project size and budget). One of the main reasons communicated for the absence of applications from several Member States is the availability of national funds. OLAF does a lot of dissemination via its various networks about the launch of the calls for proposals. At various training sessions and presentations in OLAF and Member States, the programme and its purpose is regularly highlighted. However, on the procurement side of the Hercule component, all Member States receive valuable support (through free access to commercial databases, IT analytical tools and specialised anti-fraud training).

At the same time, through its AFIS and IMS components and training for digital forensic analysts, the UAFP does reach all Member States and other eligible countries.

Figure 5: Hercule component 2021-2023 – distribution of grants by beneficiary country

Source: ICF analysis of the documentation of the Hercule component projects

3.2.4.7 Type of beneficiaries

The beneficiaries of the Hercule component are law enforcement agencies (LEAs), customs authorities, public agencies, regional authorities, universities and non-governmental organisations. The distribution of funds among these groups is shown in Figure 6 hereafter.

As expected, most beneficiaries are public agencies (such as tax administration, anti-corruption agencies and investigation units). They received 42% of the grant total and LEAs received 41%. Customs agencies were awarded 11 grants, accounting for 11% of all grants.

All grants awarded to universities and NGOs are for training, academic studies, seminars and workshops (as these type of organisations can submit eligible proposals).

Figure 6: Hercule component – grants by type of beneficiaries

Source: ICF analysis of the documentation of the Hercule component projects    

3.2.4.8End date of projects

The end date of the awarded projects is important in the context of this interim evaluation. This is because it indicates how many of the projects would have been completed by the cut-off date for the analysis of the projects’ output and impacts.

Only 14 of the awarded projects were planned to finish by the end of 2023 (of them, seven were planned to be completed in November or December 2023). Therefore, most analysis and conclusions have been based on planned objectives and output and interim results, where available, considering delays relating to COVID-19.

3.2.4.9Procured access to external databases

In addition to grants, the Hercule component of the UAFP provides funding for the procurement of equipment and services and for access to external databases, which are used by customs and other law enforcement authorities in all Member States.

In the evaluation period, access to the following three external databases was procured 70 .

-Worldwide companies information (Dun & Bradstreet). This supports customs investigations carried out by Member State authorities related to fraud undermining the financial interests of the EU. The database enables the creation of risk profiles and the verification of certain information on companies, company structures and beneficiary owners.

-Global trade statistics (IHS Global Trade Atlas). This provides import and export data, specifically the quantities and values of certain goods imported into a given Member State from non-EU countries. The database can support investigations into trans-shipment, evasion of anti-dumping or countervailing duties, misrepresentation of the country of origin, exporter or container swapping, and undervaluation fraud.

-Vessel and maritime information (source: Lloyds). This provides information on vessels and movements and supports risk analysis and intelligence related to worldwide commercial vessel operations. The database is used to cross-check the origin of goods in containers, detect potentially suspicious import transactions and risky shipments and develop risk profiles.

The total budget for the procured access set out in the AWPs between 2021 and 2023 was EUR 3 469 329, and the amount spent was EUR 2 517 760 71 . The procurement is based on framework contracts with those companies owning the respective databases.

The number of users and the usage rates were stable between 2021 and 2023. The latest user statistics (2023 data) for the three databases are presented in the table below.

Table 6: User statistics for the three databases

Source: ICF analysis of data provided by OLAF

User surveys have been carried out on an annual basis. Overall, customs and investigative authorities have pointed out that the databases are useful tools supporting their daily operations (9 out of 27 Member States responded to the 2024 survey; the survey is managed with open questions, and quantitative results are not available).

Users also indicated that the external databases provide complementary data and are used in combination with the internal applications hosted in AFIS, such as CSM, or with national databases 72 .

3.3Association agreement between the EU and Ukraine on participation

3.1.1.

3.1.2.

3.1.3.

3.1.4.

3.3.1Scope of the association

According to Commission Decision C(2024) 760 of 12 February 2024 on the approval of an agreement between the EU and Ukraine on the participation of Ukraine in the UAFP (‘the Commission Decision’) 73 , ‘Ukraine will participate as an associated country in and contribute to all parts of the programme referred to in Article 2(2), point (a) of Regulation (EU) 2021/785. This is subject to the condition that the applicable legal requirements, the nature or the scope of the actions financed allow for third-country participation.

This association does not affect the possibility for Ukraine to participate in certain operational activities under Article 2(2), points (b) and (c) of Regulation (EU) 2021/785. The conditions for such participation are governed by the applicable legal framework.’

3.3.2Terms and conditions of participation in the Union anti-fraud programme (excerpts of the related Commission Decision)

According to Article 2(1) of the Commission Decision, ‘Ukraine shall participate in the programme in accordance with the conditions laid down in Protocol III on a Framework Agreement between the European Union and Ukraine on the General Principles for the Participation of Ukraine in Union programmes and under the terms and conditions set forth in this Agreement, in the legal act referred to in Article 1 of this Agreement, as well as in any other rules pertaining to the implementation of the programme, in their most up-to-date versions.’

Article 2(2) of the Commission Decision states that, ‘unless otherwise provided for in the terms and conditions referred to in paragraph 1 of this Article, legal entities established in Ukraine may participate in actions of the programme under conditions equivalent to those applicable to legal entities established in the Union, including respect for EU restrictive measures.’

It has also been decided that English shall be used for the procedures related to requests, contracts and reports, as well as for other administrative aspects of the participation of Ukraine in the programme.

This participation entered into force retroactively on 1 January 2023, allowing Ukraine to file applications for funding, among other things, in relation to the Hercule component, covering technical assistance as well as training opportunities to be funded.

So far, Ukraine has filed applications for funding in the following areas of eligible actions of the programme:

-one application for financial support in the field of technical assistance in 2023: the application was unsuccessful, and the grant was not awarded due to a lack of sufficient budget;

-five applications for financial support in the field of technical assistance in 2024: following the evaluation of the applications and the scoring, one is considered eligible for funding under the 2024 call.

3.4Eligible actions of the UAFP – Hercule component

Only actions implementing the objectives of the programme are eligible for funding. Regardless of any other action provided for in the AWPs under Article 11 of the UAFP Regulation, the following actions may be considered eligible for funding (in 2024):

-provide technical knowledge, specialised and technically advanced equipment and effective IT tools leading to closer transnational and multidisciplinary cooperation and cooperation with the Commission;

-improve staff exchanges for specific projects, provide the necessary support for these exchanges and facilitate investigations, in particular by setting up joint investigation teams and cross-border operations;

-provide technical and operational support for national investigations, in particular for customs and law enforcement authorities, to step up the fight against fraud and other illegal activities;

-build IT capacity in the Member States and third countries, increase data exchange and develop and provide IT tools for investigating and monitoring intelligence work;

-organise specialised training, fraud analysis workshops, conferences and studies to improve cooperation and coordination between departments responsible for protecting the financial interests of the EU.

Where the action to be supported involves the purchase of equipment, the Commission must ensure that the funded equipment is appropriate for contributing to the protection of the EU’s financial interests. This is verified before granting funds to the project by an evaluation committee that checks the submitted project proposals by applicants on, among other criteria, their eligibility and EU added value.

In addition, the appropriateness and efficient use of the equipment purchased is verified through the received (ex post) reporting by the beneficiaries to the Commission (OLAF) and by planned visits at the premises of the beneficiaries.

Such on-the-spot visits took place in the first quarter of 2024, involving three beneficiaries from a Member State. The verifications included an assessment of whether: (i) the purchased equipment was in line with the description of the action; (ii) the costs declared were in line with the invoices incurred by the beneficiary during the period provided in the grant agreement and recorded in the beneficiary’s accounts; (iii) the equipment was present at the declared location, in working order and used; and (iv) the operators were trained or certified (where relevant) and able to use the equipment.

No error was detected during these verifications. The grant beneficiaries gave positive feedback about the programme, especially related to:

-the focus on purchasing equipment and allowing a selection of state-of-the-art technology, which is needed in a fast-moving technological and fraud environment;

-the absence of requiring an EU partner, which limits the administrative burden;

-the swift assessment and award of grant files. 

3.

3.1.

3.1.1.

3.1.2.

3.5Eligible entities – Hercule component

According to Article 10(1) of the UAFP Regulation, the complementary eligibility criteria set out in the Regulation apply, in addition to the criteria set out in Article 197 of the Financial Regulation. This means that, as specified in Article 10(2) of the UAFP Regulation, the following entities are eligible under the programme:

a)    public authorities that can contribute to achieving one of the objectives referred to in Article 2 of the Regulation and are established in:

i.    a Member State or an overseas country or territory linked to it;

ii.    a third country associated to the programme;

iii.    a third country listed in the work programme under the conditions specified in paragraph 3;

b)    research and educational institutes and non-profit-making entities that can contribute to the achievement of the objectives referred to in Article 2 of the UAFP Regulation, provided that they have been established and have been operating for at least 1 year in:

i.    a Member State;

ii.    a third country associated to the programme;

iii.    a third country listed in a work programme under the conditions specified in paragraph 3;

c)    any legal entity created under EU law or any international organisation.

Moreover, according to Article 10(3) of the UAFP Regulation: ‘Entities referred to in paragraph 2 established in a third country that is not associated to the programme are exceptionally eligible under the programme where this is necessary for the achievement of the objectives of a given action. Such entities in principle bear the cost of their participation, except in cases that must be duly justified in the AWP’.

3.1.3.

3.6Implementation of the Irregularity Management System (IMS) and the Anti-Fraud Information System (AFIS)

Like AFIS, IMS is not a financing programme supporting grantees’ projects. Its main role is to facilitate reporting detected cases of irregularities and of suspected or established fraud in the implementation of EU funds. The system is used by EU Member States and other beneficiaries (spread over 34 countries and used by 789 organisations and over 3 230 registered users as of March 2024) 74 .

IMS is perceived by users and stakeholders as a useful reporting and tracking tool having, in particular, on the one hand, the following strengths:

-it is the only IT system in the EU where data on past and closed cases of irregularities are collected and can be searched;

-IMS provides contextual information on irregularities and on the state of the fraud identified in the Member States, candidate countries and third countries;

-IMS provides a standardised tool to collect data on irregularities;

-most users find the system reliable and user-friendly, highlighting features like the visibility of data, the ability to download it and the availability of data in all EU languages;

-the IMS interfaces with national databases and can support automatic data transfer.

On the other hand, the evaluation identified certain IMS weaknesses related to technical properties and the management of the system, and also to the way users were entering data in the system. Additionally, there is a need to ensure the quality of data because not all cases were entered into the system (which obviously depends on the identifying partner country) and not all information on the cases was available (completeness, depending on the partner again). The input of data by different users in distinct Member States, regions, etc. was not always consistent, homogeneous or coherent (reliability and consistency). The data were not always up to date with the most recent developments (promptness).

The technical weaknesses that were mentioned, i.e. issues related to authentication, difficulties when searching for information and time-consuming checks of irregularities, insufficient communication on updates and improvements to IMS, and insufficient guidance on which information to enter, have largely been tackled by the very recent October 2024 IMS release, striving to solve the mentioned issues in the near future (as training and further guidance is still ongoing).

There are no penalties for countries that fail to fully report. This may – to a certain extent – explain the system’s lower rate of use by contributing authorities. OLAF investigators and selectors and Commission users from shared management programmes also use data from the IMS to a relatively low extent. This stems from the issues already mentioned as well as a lack of awareness of how the system works and what it can offer.

3.6.1Implementation – 2023

The IMS operates under the AFIS platform. The common information system requires stable financing over time to ensure its sustainability.

The UAFP finances the IMS. Through this component, the UAFP allocated EUR 7 000 000 for 2021-2027 and EUR 985 119 for 2024. The aim is to support reporting irregularities, including fraud, of the shared management funds and pre-accession assistance funds of the EU budget.

The implementation is reported on for the most recent period, in this case 2023, as reported in 2024 by the Commission.

3.6.2Main activities 

To protect the EU’s financial interests, EU law requires Member States managing EU spending under shared management and countries benefiting from pre-accession assistance to notify the Commission of any irregularities (including suspected and proven fraud) detected in areas where the EU provides financial support (see list in Chapter 2.1.1.5).

To facilitate the reporting of irregularities, the IMS has been developed and put at the disposal of the Member States and other beneficiary countries. This dedicated electronic system is being used by 34 countries.

Member States, candidate countries and other non-EU countries have set up a hierarchical reporting structure with multiple levels of responsibility. Around 750 reporting organisations, covering over 3 000 IMS users, are responsible for the timely reporting of irregularities.

The reporting workflow includes multiple hierarchical levels and various roles within the same level to ensure that several quality checks are performed before the reports are sent to the Commission.

Reporting authorities provide information on:

-who committed the irregularity/fraud (persons involved);

-the support measure such as fund, programme, project, budget line;

-the financial impact (expenditure and irregular/fraudulent amount);

-how the irregularity/fraud was committed;

-when the irregularity/fraud was committed;

-where the irregularity/fraud was committed;

-how the irregularity/fraud was detected;

-what administrative, judicial or penal follow-up sanctions were imposed.

3.6.3IMS – budgetary implementation

The 2023 AWP had a budget of EUR 959 783 dedicated to IMS maintenance, IT development and studies. The full available amount for 2023 was committed. The total indicative budget for this component for 2021-2027 is EUR 7 million.

3.6.4Maintenance and development

In 2023, eight releases were launched. Features included allowing IMS users to manage and monitor EDES 75 -hits, adding case summaries and descriptions of codes, improving the B2B services and fixing a number of identified issues.

3.7IMS developments

Overall, the IMS is perceived by users and stakeholders as a useful tool for reporting and tracking irregularities and fraud. The strength of the IMS lies in its facility to search for past and closed cases, which enables tracking irregularities. Another strength of the IMS is that the system is considered to be reliable and provides users access to the information they enter.

The IMS is the only EU system where such data are collected. For some EU Member States, it is also the only tool to gather such data and where that data can be consulted. This means that it is a unique system, sometimes even at national level. The data gathered provides the main source of knowledge about the scale and impact of detected fraud and irregularities tracked and reported through annual PIF reports. 

Some users highlighted challenges related to authentication, difficulties in searching for information, time-consuming checks of irregularities, insufficient communication on updates and improvements in the IMS, insufficient guidance on which information to enter, the need to better reconcile information in the IMS with other IT tools, and the need to ensure the quality of data.

In addition to the challenges with how the tool itself works, broader issues connected to the tool relate to a lack of sufficient knowledge and training among some users preventing them from using the tool effectively.

There is also a need to modernise certain aspects of the IMS user interface to improve the overall user experience. Furthermore, there was a relatively low usage rate of this data across OLAF investigators and selectors (who evaluate the merits of new incoming information for opening investigations) and Commission users from shared management programmes. This low rate was attributed to a low awareness of what the system is and does, how to use it and find the cases of interest in the system, and how to reconcile the information with other data sources and IT systems.

In the respective annexes to the annual PIF reports published, figures do not include third countries (including pre-accession countries) or direct expenditure irregularities.

The number of irregularities reported as ‘fraudulent’ measures the results of Member States' work to counter fraud and other illegal activities affecting the EU's financial interests. Therefore, the figures should not be interpreted as indicating the level of fraud in Member States.

In 2019, 11 726 irregularities were reported to the Commission (OLAF), involving a total amount of approximately EUR 1.6 billion. Of these irregularities, 939 were reported as fraudulent, representing an amount of approximately EUR 444 million (not taking into account the amount reported as fraudulent by the UK, which was about EUR 2.3 million) 76 .

In 2021, 10 232 irregularities were reported to the Commission, involving a total amount of approximately EUR 3.2 billion. Of these irregularities, 948 were reported as fraudulent, representing an amount of approximately EUR 1.8 billion for EU-27 77 . This much higher amount includes an amount of EUR 1.4 billion on the expenditure side, reported by one Member State in 2021.

In 2023, 12 240 irregularities were reported, involving a total amount of approximately EUR 1.8 billion (EU-27). Of these irregularities, 988 were reported as fraudulent, representing an amount of approximately EUR 564.3 million for EU-27 78 .

Between 2021 and 2023, the overall number of reported irregularities increased by 19.6%, after having decreased between 2019 and 2021. Between 2021 and 2023, the number of irregularities reported as fraudulent increased by 4%.

3.8The Anti-Fraud Information System – Implementation

Unlike the Hercule component, AFIS does not provide applicants with financial support to develop projects. Its main objective is to facilitate the exchange of fraud-related information between national and EU administrations.

The effectiveness of its implementation can be evaluated by its output (e.g. the number of new applications developed, the number of upgrades performed or features added and the number of mutual assistance activities, such as joint customs operations supported).

A key performance indicator for AFIS is the number of information items on mutual assistance made available. For assessing the impact of AFIS output, the key indicator is the level of satisfaction of AFIS users as measured by periodic user surveys. The results from the latest survey in 2023 are compared with the results from the previous one (in 2019) in the table below.

The AFIS budget performance is determined by the OLAF-set indicator, which reflects the amount of mutual assistance information made available and the number of supported mutual assistance-related activities. For 2021-2023, the targets for this indicator were exceeded. The AFIS budget also contributed to improving mutual administrative assistance among customs authorities in the EU. It supported, between 2021 and 2023, a total of 35 joint customs operations and helped organise training sessions for key AFIS applications, such as CIS, VOCU, CSM and AMT 79 .

3.9Implementation of the UAFP in 2023

The implementation of the programme is reported on for the most recent period, in this case 2023, as reported in 2024 by the Commission.

Through this third component of the programme, the UAFP allocated EUR 60 million for 2021-2027 to AFIS, of which EUR 8 445 091 has been allocated for 2024. The aim is to provide tools for information exchange and support for operational activities in the field of mutual administrative assistance in customs and agricultural matters.

This component of the UAFP also covers the cost of using the Customs Information System (CIS) provided for in instruments adopted under Article 87 TFEU, in particular in Decision 2009/917/JHA, given that those instruments state that such expenditure is to be borne by the general budget of the EU. The Customs Information System is an automated information system. It helps Member States prevent, investigate and prosecute activities that are in breach of customs or agricultural legislation. It does so by increasing, through faster dissemination of information, the effectiveness of the cooperation and control procedures of the customs administrations whose remit covers such activities.

The AFIS platform launched 44 releases, consisting of several combined application releases. A new version of the Import, Export and Transit directory (IET) and several other application releases and fixes were developed in 2023, totalling more than 110 releases. Among these were several major releases:

-a new version of the Import Export and Transit directory that allows users to carry out advanced searches, set alarms and export data (e.g. in Excel format) on imports, exports and transits;

-two new versions of the Container Status Message directory implementing visual analytics for container trips and the new Container Origin Signals module, which signals potential origin fraud;

-an improvement to the anti-fraud analytics platform, providing extended computing resources, improved access, added data sources and enabling OLAF analysts to combine customs and trade data from different sources and conduct advanced analysis.

In 2023, AFIS also provided technical support to 13 joint customs operations’ meetings.

The 2023 user survey confirmed that AFIS remains a useful tool for its end users, and that AFIS is able to deliver functioning tools for information exchange in its dedicated area of action. Two thirds of the survey respondents believed that the applications available in AFIS are up-to-date tools that help tackle the latest trends in fraud and related irregularities.

The percentage of users who are satisfied with the functionality and performance of the various AFIS applications remains high (with the most frequently used applications scoring satisfaction rates above 80%). There are some minor differences compared with the results of the 2019 AFIS satisfaction survey (see Table 7 hereafter).

Table 7: Comparison of the 2023 AFIS satisfaction survey with the 2019 survey (key applications)

Source: AFIS satisfaction survey 2019 and 2023 (N=1320)

3.10AFIS – developments in the past decade 80

As AFIS is not a financial programme, it has not been subject to the corresponding (programme) evaluation cycles. Nevertheless, there is a set of indicators that can help evaluate the performance of the related operational activities 81 .

In particular, OLAF's Internal Audit Capability (IAC    ) conducted an audit to assess the performance of AFIS in 2013 and 2014. Its 2014 report 82 confirmed the efficiency and effectiveness of AFIS and the adequacy of the management and coordination mechanisms. It concluded that stakeholders considered the quality of AFIS services as satisfactory to highly satisfactory.

Overall, already back in 2014, the operational activities under AFIS were a valuable support to Member States' and the Commission's operational and investigative work, which aimed to detect customs infringements, recover customs duties and improve customs cooperation in the anti-fraud area. The efforts made before in addressing identified shortcomings have contributed to increasing the level of trust, participation and satisfaction of Member States' competent authorities.

More specifically, the permanent physical (Permanent Operational Coordination Unit – POCU) and IT infrastructure (Virtual Operation Coordination Unit – VOCU) for the support of JCOs has been regularly used by the Commission, Member States, third countries and organisations on many occasions.

The evaluation reports submitted after the completion of JCOs show that this physical structure and AFIS application strongly facilitate the coordination tasks associated to JCOs with a large number of participants. Their use is frequently recommended in the evaluation of these operations. The fact that these facilities are also frequently used for operational activities in the area of law enforcement cooperation (free of charge for the participants) is additional proof of Member States' appreciation of the AFIS.

Importantly, covering the expenses for transport, accommodation and daily allowances for representatives from Member States attending training courses and meetings related to investigative and operational actions, including JCOs, has proven to be critical in securing the participation of a large number of Member States in these initiatives 83 .

Following Member States’ recommendations provided at workshops organised by the Commission, OLAF has strengthened, between 2014 and 2018, the involvement of Member States in all stages of development of new or updated AFIS applications, aiming to improve its use and user satisfaction. This was the case during the update of AFIS VOCU 84 , AFIS Mail 85 , Customs Information System (CIS) 86 and IMS 87 , which involved a large number of Member State participants in the above-mentioned workshops. As a result of this collaboration, Member States’ acceptance and use of these applications significantly improved, along with an increase in the size of these user communities.

In terms of AFIS and IMS, as IT platforms being directly managed by OLAF, they were not subject to the same evaluation process as the Hercule III programme (as they were not part of the programme at the time). As such, there is limited data on their use in the years before their integration into the UAFP. Nevertheless, some evaluative evidence exists.

Results from the 2019 AFIS satisfaction survey showed that most of the applications were evaluated positively by the participants of the survey in terms of both functionality and performance, with satisfaction rates often exceeding 80% 88 . On the available training options, one third of the participants declared that they used at least one of the training options on the AFIS Portal, and most of them (more than 67%) identified the user portal manuals as their main training source for the AFIS Portal and its applications.

Figure 7: 2019 AFIS survey: Distribution among the AFIS applications based on measuring their regular use

3.11AFIS budget evolution between 2014 and 2023

The total indicative budget for this component for 2021-2027 is EUR 60 million. The 2021 budged reached EUR 8.89 million and was 19.3% higher than in 2020 due to specific IMS funds included in 2021.

In 2021, AFIS was allocated an amount of EUR 7 964 000 for the first working year. However, from this budget allocation, EUR 2 775 704.46 had already been committed under Commission Decision C(2021)2120 of 7 April 2021 on the interim financing of the AFIS and the adoption of its 2021 work programme 89 .

That Decision was then repealed with effect from 17 May 2021 (the day on which the Regulation establishing the UAFP entered into force) but did not affect the actions initiated under that Decision. This legal arrangement was necessary to cover the financing of the AFIS between 1 January 2021 and 29 April 2021 when the Regulation was adopted.

For 2021, this left a budget of EUR 5 188 295.54 available for the remainder of the 2021 AFIS work programme. EUR 8 445 091 has been allocated to AFIS for the year 2024. This is an increase of 6.04 percentage points, compared to 2021.

4Evaluation findings (analytical part)

This Chapter presents a summary of the main evaluation findings, following the analysis and triangulation of the evidence collected by the external study by ICF. To keep the reporting here succinct, the questions, responses and analysis are described in full detail in Annex III.

4.

4.1.To what extent was the intervention successful and why?

This section summarises the analysis presented in the study in relation to the three criteria of effectiveness, efficiency and coherence, covering the three programme components.

The intervention is considered to have been, within the budgetary limitations, (very) successful during the period covered by this evaluation. The evaluation study has been able to confirm that the technical assistance (purchase of equipment) and training actions, as well as procured activities, that have been funded and provided under the UAFP, have been effective or could still be effective in improving the prevention, detection and investigation of fraud and other illegal activities detrimental to the EU's financial interests.

So far, the study has indicated that activities supported under the three components of the UAFP make the following indirect, yet successful contributions to limiting the exposure of the EU’s financial interests to fraud, corruption, and other illegal activities.

In relation to the ‘Hercule’ component, it was demonstrated that the technical assistance projects boost the capacity of enforcement agencies to investigate or prevent the known exposure by providing up-to-date technological and knowledge resources to combat fraud and irregularities. With regard to the training actions, publications, and studies supported by the UAFP contribute to improving the capacity of customs and enforcement agencies’ staff and developing the knowledge base needed for combating fraud, corruption and other illegal activities.

The evaluation has shown that activities supported by the UAFP have contributed to a significant extent to the improvement of the beneficiaries’ work (claimed by 77% of consulted beneficiaries). Beneficiaries also point out that the programme has significantly contributed to the improvement of their investigative capacity (84%), operational capacity (69%) and technical capacity (71%).

All consulted beneficiaries agree that the projects supported by the programme have contributed to them acquiring new skills and knowledge of specialised methodologies.

The evaluation has allowed for a number of challenges to be identified during the implementation of the programme, described as follows:

-The Commission’s application platform, particularly the reporting platform, were perceived by some beneficiaries as not being sufficiently user-friendly. In general, beneficiaries complained that they had to call technical support to understand how the platform works.

-Potential applicants are not automatically alerted electronically when a new call is published. Several groups and networks of stakeholders are informed by OLAF through different communication channels when calls are published and are asked to send the information to their colleagues and members in their organisations. Beneficiaries appreciated, however, that the calls follow a regular cycle.

-Beneficiaries have expressed an interest in being informed about similar projects that are being implemented. This request was addressed in 2022 and 2023 when OLAF organised workshops where beneficiaries attended and also presented their own projects.

-In projects where advanced technology is to be procured, between the time of application and the time of procurement, technological advancements may make the technological solution envisaged in the application less useful. Prices may also change significantly during this timeframe.

-The importance of ensuring, as much as possible, interoperability between the AFIS applications and with other systems of customs relevance, in particular the support of searches across different AFIS databases and systems, was particularly underlined by Member States. In the specific field of analysis, the creation of an AFIS data lake to enhance the exploration of the information stored in the various AFIS customs applications was recommended.

-A number of possible (AFIS) improvements were indicated, such as the creation of working groups and the organisation of workshops dedicated to the sharing of experiences and best practices, the need for additional on-line AFIS training materials and training packages for specific purposes, as well as the organisation of AFIS trainings on a more regular basis, including training dedicated to specific areas such as analysis.

The evaluation, also based on the 2023 AFIS user survey, confirmed that AFIS remains a useful tool for its end users, and that AFIS is able to deliver functioning tools for information exchange in its dedicated area of action. The evaluation has shown that AFIS supports customs officers in the timely exchange of up-to-date information to prevent fraud.

As for IMS, this tool supports successfully the timely exchange and sharing of irregularities information, enabling its users to use agencies’ past experiences in drawing up fraud preventive measures. The 2023 AFIS satisfaction survey contained questions on the functionality and performance of IMS for which the satisfaction rate was approximately 91% 90 . This compares very favourably to the 2019 results, showing an increase now with 19 percentage points.

The IMS and its associated specific objective have been relevant to achieving the general objective of protecting the EU’s financial interests as the IMS is showing encouraging signs of being a useful tool for Member States to report irregularities and contribute to the broader fight against fraud.

At the same time, interviews with several IMS country managers revealed certain areas for improvement. For instance, some forms have too many fields, and these fields sometimes have confusing descriptions.

The ongoing upgrading of IMS (Q4 2024) has not yet been able to identify a measurable improvement to the user experience yet, while users are increasingly expected to report more data and to improve the quality of the existing data. These issues may also affect their satisfaction levels. This is not purely an OLAF/IMS issue but reflects a general greater ‘overall’ demand for data to be registered, reported and exchanged for various purposes, including accountability.

With regard to the evaluation of the programme in relation to the efficiency criterion, an integral benefits and costs analysis has been presented in detail in Annex IV.

The analysis done suggests that the implementation of, and the participation in the programme can be considered to be an efficient use of resources based on a (partial) cost-benefit analysis. However, the data available at this interim stage of the programme does not make it possible to produce a complete, quantitative mapping of programme costs and benefits or to monetise all costs and benefits for direct comparison with one another. For this reason, the comparison between costs and benefits has been primarily made (in the external study) on a qualitative basis.

Concerning the efficiency of the administrative processes of the Hercule component, three sub-sections can be considered, i.e. the application procedure, the project implementation period and the budgetary flexibility.

The application procedure for grants under the Hercule component of the current UAFP is found to be highly efficient based on consultations with applicants. The set-up of the current application process shows it has taken on board the feedback received during the final evaluation of the Hercule III programme (end of 2021).

The final evaluation suggested some improvements to make the application process more efficient for the benefit of the applicants. The final evaluation found the application procedures more efficient than the previous evaluation period (mid-term evaluation of 2018) because applicants took less time to prepare their application, compared with the previous Hercule programmes. However, the 2021 final evaluation recommended improving the application process by providing additional and more refined guidance to applicants on how to adequately fill in the application form because applicants had incurred human resources costs, allocating civil servants or staff to proceed and help with the application 91 .

This cost aspect (time and human resources) of the application procedure, however, was not found to be as severe as to deter applicants from applying to the programme. Nevertheless, the 2021 Hercule III final evaluation recommended providing sufficient information to applicants (Recommendation 9.1), such as through best practices and example boxes directly in the online application form. It also recommended setting up a platform or appointing a contact person to answer questions (Recommendation 9.2) and organising an annual workshop or seminar focused on the application process (Recommendation 9.3). Lastly, it recommended that the application procedure should avoid ambiguous questions (Recommendation 9.4).

These recommendations have since all been implemented in the current UAFP. Most of the applicants 92  found the guidance and instructions provided to prepare the application clear and readily available 93 . The call documents 94 , published under each annual work programme, provide a clear overview of the projects eligible for funding and present the eligibility criteria clearly. Furthermore, each call is now accompanied by an online manual for proposal preparation and submission, which is easy and intuitive to navigate. 

Applicants can also receive support if they have technical difficulties, through the Portal Submission System, as well as with specific non-IT related questions for which two different email addresses are available. Only a minority of stakeholders would have liked to see even more guidance: this is particularly the case for researchers who stated they had not received adequate support from their own institution when preparing the application, which is beyond the Commission’s influence or control.

In terms of procedural efficiency, the application procedure has proven to be efficient by avoiding requesting the same information several times throughout the application process: most survey respondents stated that they were not asked to input the same information several times, which would have added unnecessary administrative and time-consuming steps 95 . 

The fact that the application form and process require information relevant to the objective of evaluating the project (therefore avoiding unnecessary and resource-consuming steps, documents or additional administrative steps) confirms the high efficiency of the application 96 . Therefore, the administrative and financial requirements are overall proportional and reasonable 97 . 

The in-depth interviews with stakeholders have revealed some points for further improving the application process in the Hercule component (technical assistance and training part). First, accessibility and inclusion needs to be improved. Some applicants with visual impairments, for example, are unable to access the application platform because Oracle platforms do not support all screen-reading software. Although the portal is very easy to use for most people, the general ease of access for people with disabilities should be examined (although this is a technical feature, which does not fall under the remit of OLAF) 98 . 

Second, several stakeholders indicated that the application process should be tailored to how large and complex the project is, which would simplify the procedure for smaller projects 99 . This request may however encounter some technical issues with regard to the existing templates for filing an application, as well as issues of equal treatment.

The proportion of successful applicants increased considerably between 2021 and 2023, implying that less time is lost by applicants on unsuccessful applications 100 . This increase in efficiency may have been driven by the procedural changes to the online application process and by the two workshops organised by OLAF in 2022 and 2023 with potential applicants and past beneficiaries (the workshops included updated guidance on the application process).

With regard to the second sub-section of the project efficiency evaluation, namely the ‘Hercule’ project implementation period, half of the beneficiaries of technical assistance (TA) under the UAFP Hercule component, who were surveyed for this evaluation, reported that UAFP funding has contributed to their project achieving notable outcomes and results 101 . This finding needs however further validation, which should be through evidence from any available data on the (later) reported results of the finalised projects and their costs of implementation.

In this context, the Commission (OLAF) refers to the examples given in the specific annex to the annual PIF report, i.e. the ‘Annual Overview on the implementation of the UAFP’ (per year).

The financial support provided via the TA component has so far led to significant or very significant cost savings in a third of the cases and to some cost savings in almost half of the cases 102 . In terms of resources, in a minority of cases, the funds and resources made available were not enough to support implementing the action 103 . This may have been due to budgetary planning or other factors, which have not been possible to identify based on the available data.

The grants received have allowed agencies to increase their operational and investigative efficiency. The purchase of equipment, for example, allows them to save time in extracting and analysing data for criminal proceedings 104 . Therefore, beneficiaries noted that the improved capacity and capabilities will not only allow them to be more efficient in carrying out operations in their countries but will also make them better equipped to respond to requests for mutual assistance and joint investigations with EU partner agencies 105 . 

As for the programme’s budgetary flexibility, the grant agreement gives beneficiaries a good degree of budgetary flexibility 106 . After receiving the grant, applicants can request to make changes to the budget breakdown by transferring budgetary resources between participants and between budget categories (as long as this does not involve any substantive or significant change to the description of the action) 107 . Interviewees who have made use of the budgetary flexibility were satisfied with this aspect of the UAFP, which allowed them to change some aspects of the project without having to complete a formal project amendment 108 .

The budgetary flexibility in these cases increases the projects’ efficiency since it reduces the administrative burden for beneficiaries. Commission (OLAF) programme managers and financial officers do note an increasing workload due to a higher number of requests for changes within projects and/or of their budgets in recent years.

A third and final evaluation criterion tackled under this first evaluation question on the successfullness of the programme, is the assessment of the programme’s coherence. This assessment focuses on two aspects, namely the internal and the external coherence (or consistency) of the UAFP and its interventions.

The internal coherence assessment examines the extent to which the three components (Hercule component, AFIS component and IMS component) and the different types of actions of the UAFP are sufficiently clear and coherent with one another and do not contradict but rather support each other’s implementation by creating synergies.

The external coherence or consistency explores the extent to which the interventions of the UAFP are in line with other relevant legislative and policy instruments and interventions taken at EU level. Several other EU programmes mentioned below were selected for the in-depth analysis of coherence, as indicated in the following sections.

The coherence assessment has been based on the results of desk research (external study), including the analysis of the UAFP Regulation and other programmes with similar objectives, and is informed by the analysis of stakeholders’ views (interviews and surveys).

In particular, interviews were carried out with beneficiaries and with a number of Commission departments to gather input on internal coherence and the external coherence of the UAFP with other EU programmes, focusing on potential duplication (financial overlap/double funding) and synergies (complementarity).

Information provided by stakeholders on coherence based on the survey for applicants – including beneficiaries – was cross-referenced for this assessment.

The research also looked at the calls for proposal and the projects funded under the UAFP and other EU initiatives. However, their broad description does not make it possible to highlight any major duplication, assuming they exist.

Based on the analysis of the relevant programmes as well as the results of the stakeholder consultation conducted for the evaluation, the UAFP is considered to be coherent (internally) and consistent (externally). Nevertheless, some overlaps and areas for improvements are identified (as indicated in the following sections).

The external coherence or coordination assessment focuses on the extent to which the UAFP and interventions implemented through the UAFP are coherent with other instruments and interventions with similar objectives. In addition, it also examines the coherence with EU horizontal policies.

On the topic of coherence with (other) EU horizontal policies, and as indicated in the UAFP AWPs, implementation of the UAFP is intended to be in line with the EU’s political priorities 109 and related policy initiatives.

Among the cross-cutting policies, the UAFP reflects the EU’s commitments 110 to tackle climate change in line with the Paris Agreement adopted under the United Nations Framework Convention on Climate Change (UNFCCC) 111 .

The programme aims to contribute to mainstreaming climate action and helps, where possible, in achieving an overall target (for the Commission) of 30% of the EU budget supporting climate objectives 112 . 

During the consultation for the study, it was reported that applicants are encouraged to highlight proposed activities in their applications that take into consideration the climate objectives of the EU. Funding under the UAFP was provided to beneficiaries to purchase energy-efficient equipment, such as specialised electric or hybrid vehicles 113 . 

The UAFP might also be used to better equip Member States to prevent imports of certain illicit products that do not comply with the EU environmental and climate framework 114 . Despite these efforts, some stakeholders stressed that the size of the UAFP is limited and cannot significantly contribute to addressing climate change concerns 115 . 

In addition, it was highlighted that, although UAFP funding is given to projects that aim to be in line with energy-efficient standards, the actual implementation of the projects might not be in line with such standards. This is because public procurement processes put in place by national administrations often favour the lowest-cost bidder over other considerations.

The digital transition is another key priority of the EU and part of its digital strategy 116 . The UAFP supports Member States’ digital transitions through all three components.

For instance, this already partly occurs through the UAFP funding to Member States’ beneficiaries for the purchase of IT tools supporting data analysis to detect fraud affecting revenue or expenditure and by building up expertise on digital forensics 117 . Financial support is also provided to purchase and maintain software and hardware that is needed to improve the IT capacity of the beneficiary and to strengthen its organisation, including judicial and law enforcement capabilities.

The financial support mentioned above (funding of IT related equipment and tools, plus the budget allocated to AFIS and IMS) is considered a 100% contribution to the UN SDG Number 16 – Peace, justice and stronger institutions. The UAFP contribution to this digital transition was estimated at around EUR 11.6 million in 2021 and around EUR 11.94 million in 2022. EUR 16.4 million has been allocated in 2023 to run and manage IMS and AFIS, including also a number of grants for technical assistance projects in the field of IT and digitalisation of law enforcement agencies. The latter projects will continue to run and have an impact in 2024 and 2025.

This funding is stemming from the overall annual budget of the programme, which typically ranges between EUR 23-24 million per year and includes the AFIS and IMS budget. The AFIS and IMS budget are considered to fully boost the programme’s IT capacity building of its beneficiaries.

The programme indirectly helps building effective, accountable and inclusive institutions in Member States at various levels of national and/or regional administrations (law enforcement agencies), in particular through support given, leading to an enhanced digital transition.

However, given the limited size of the UAFP budget, it was stressed that this contribution would not be significant enough to facilitate fully reaching the digital transition objectives 118 .

Regarding the coherence of the UAFP with other single instruments and interventions, as noted already in the mid-term evaluation of the Hercule III programme, most of the stakeholders consulted at that time reported more synergies than potential overlaps with other EU-funded programmes. However, they indicated more coordination was needed in the future with other programmes managed by DG TAXUD and DG HOME, covering the fight against VAT fraud and against corruption or other organised crime types 119 .

The consultations revealed that OLAF is exploring the possibilities of expanding the scope of its investigative work in new areas, including, food fraud, chemical waste, waste shipment and other environmental issues. For this study, the potential coherence of OLAF's expanded scope with financing programmes managed by other Commission departments could be explored in future studies.

For the external coherence analysis, several key EU programmes were selected in the study (see Box 1 hereafter). Only the two closest related ones are described in detail hereafter 120 .

Overall, the coherence with the other EU programmes is confirmed by the results of the desk research during the study and by most of the stakeholder interviewees. In particular, one beneficiary underlined the unique characteristics of the UAFP, which is recognised as filling a critical gap by funding projects that would not otherwise be covered by other EU funds 121 .

Box 1 - Key instruments and interventions for external coherence analysis

The UAFP is also aligning more and more with the 2019 Commission’s anti-fraud strategy, which places more attention on the collection and use of data for anti-fraud purposes 122 .

Interventions financed under the UAFP aim to focus on cross-border cooperation in fighting fraud and to ensure there is complementarity and interoperability with equipment and tools purchased under other EU-funded programmes 123 .

Existing mechanisms aiming to prevent (financial) overlapping between the UAFP and other financial instruments

Mechanisms, consultations and cooperation measures bring together the various stakeholders involved to prevent financial instruments’ objectives from overlapping.

Formal and informal mechanisms are in place to facilitate cooperation across the Commission’s departments, prevent or resolve potential overlaps and ensure optimal synergies between EU-funded programmes.

The formal cooperation takes place during the preparation of the AWPs for the different funding instruments. The programmes and their legislative activities (adopting the financing decisions and annual or multi-annual work programmes) are subject to inter-service consultations (internal cross-departmental consultations in the Commission).

The legislative proposals are submitted to all relevant Commission departments for their input and agreement to ensure coordination between the programmes managed by those departments.

The Commission departments also coordinate their work for the launch of calls for applications and during the evaluation of the applications received. The applications are examined to identify any overlaps and to prevent over- or double-financing.

In practical terms, this means that OLAF staff are part of the different evaluation committees set up each year for the assessments of the applications received under other programmes. Similarly, staff of other Commission departments participate each year in the evaluation committee for the assessment of applications submitted for UAFP funding.

In addition to the formal mechanisms above, there are informal channels of cooperation for exchanges at the level of policy officers of the relevant units. This level of informal cooperation is considered essential in leveraging the in-house expertise on the differences in scope and areas of competence between the UAFP and, for example, DG TAXUD’s programmes 124 .

Strengthened and regular coordination is ensured between DG TAXUD and OLAF 125 , as well as between DG JUST and OLAF to avoid overlaps in providing grants and training to similar target audiences 126 .

At a higher level, there is also cooperation when developing and adopting the legislation setting out the instruments (e.g. DG TAXUD contributed to the impact assessment prepared ahead of the UAFP Regulation) and when evaluating their implementation and results 127 .

The study provided a comparative analysis of the UAFP and related EU programmes, of which two are described hereafter, i.e.  Regulation (EU) 2021/1077 establishing, as part of the Integrated Border Management Fund, the instrument for financial support for customs control equipment (CCEI), and Regulation (EU) 2021/444 establishing the Customs programme for cooperation in the field of customs (‘Customs programme Regulation’)  128 .

The CCEI is one component of the EU’s Integrated Border Management Fund (BMF). For 2021-2027, it aims to contribute to adequate and equivalent customs controls by supporting Member States in purchasing, maintaining and upgrading state-of-the-art customs control equipment.

During the interviews with Commission’s officials, the highest risk of duplication in funding was identified as existing between the UAFP and the CCEI. In fact, the two programmes have similar eligible activities. The UAFP’s scope covers the protection of the EU’s financial interests, which is also one of the areas covered by the CCEI but not its main priority. In addition, the UAFP comprises of activities related to customs, which is the sole area covered by the CCEI. The UAFP includes customs authorities among its beneficiaries, which are the only specific beneficiaries of the CCEI.

However, although the CCEI has a much larger budget than the UAFP, its scope is more limited than the UAFP’s since the CCEI focuses on customs equipment specifically. The complementarity between the two programmes was already identified at the proposal stage of the UAFP Regulation. However, the Regulation indicated that each of them focuses on different types of support, namely to national authorities and customs authorities 129 . 

Regarding technical assistance support, the adoption of the CCEI allowed for the UAFP to focus more on advanced tools and technologies, including data analysis. This created valuable synergies (complementarity) across the two programmes, while also avoiding overlaps in the funding, including financial overlaps 130 .

To limit and avoid such duplication, OLAF and DG TAXUD provided guidance to applicants and beneficiaries to help them distinguish between the UAFP and the CCEI. Efforts were regularly made to ensure closer and more effective cooperation and consultation between OLAF and DG TAXUD. This was carried out via formal cooperation (such as inter-service steering group meetings and joint meetings at different levels) and informal cooperation (including regular and ad hoc exchanges between policy officers) to avoid duplication and optimise coordination.

Consultation took place at different stages, in particular when preparing the relevant regulations (e.g. with DG TAXUD and other Commission departments providing input to the UAFP impact assessment), developing the AWPs, launching the calls for applications and evaluating the applications received (e.g. to avoid double-financing).

Regulation (EU) 2021/444  establishes the Customs programme to further modernise the Customs Union, support the development and uniform implementation of customs legislation and policy and facilitate cooperation in the field of customs for the 2021-2027 financial period. It includes funding for collaborative activities, administrative and IT capacity building, including human skills and training, and the development and operation of EU electronic systems and innovation in customs policy.

The above-mentioned Regulation indicates that the Customs programme should exploit possible synergies with other EU measures in related fields, such as the UAFP, in order to ensure cost effectiveness.

The external study 131 identified common points and areas of potential overlap between the UAFP and the Customs programme, as presented in the table hereafter. These common points raised questions with the interviewees during the study about their differences and how to avoid duplicating efforts in the customs area 132 .

Table 8: Scope for coherence and overlap between the UAFP and the Customs Control Equipment Instrument (light pink colour)

One of the main common points is that they both aim to contribute to the protection of the EU’s financial and economic interests and may address the same customs and trade policy topics (as part of the general and specific objectives of the two programmes). Although this is common ground, the nature and scope of the two programmes is clearly distinguishable. The Customs programme focuses solely on customs cooperation and customs union aspects, whereas the UAFP supports the fight against fraud, which is not in the scope of the Customs programme. The UAFP is not specific in terms of the law enforcement aspects and, as such, goes beyond the customs domain.

In terms of eligible actions, both programmes can finance electronics and IT systems. They also both can support organising specialised training, risk analysis workshops, conferences and studies that aim to improve cooperation and coordination. However, the Customs programme cannot do so if the objective goes beyond its legal scope 133 .

Regarding the technical assistance provided under the UAFP, the UAFP is used to fund the purchase of highly specialised technical equipment with possible uses that are not only limited to customs activities, with OLAF focusing increasingly on the expenditure side of the programme. The purchase of similar technical equipment is excluded under the Customs programme.

Considering the common points and to avoid overlaps, DG TAXUD and OLAF work very closely together (formally and informally) to coordinate and build complementarities. A clear understanding has been reached between the relevant Commission departments on the differences between the two programmes 134 . They have also engaged actively with applicants and beneficiaries to provide guidance on how the two programmes work and what are their specific objectives. The similarities were already evident at the time of the Hercule III programme. However, cooperation made it possible to tackle similar policy aspects in different ways as confirmed by stakeholders during the final evaluation of Hercule III 135 . 

1.

2.

3.

4.

4.1.

4.1.1.

4.1.2.

4.1.3.

4.1.3.1.

4.1.3.2.

4.1.3.3.

4.1.3.4.

4.1.3.4.1.

4.1.3.4.2.

4.1

4.2How did the EU intervention make a difference and to whom?

This section, focussing on EU added value provided by the programme, assesses the potential change for beneficiaries or the public as a result of the EU intervention and measures the impact above and beyond the reasonably expected results of national actions alone.

Due to the limited availability of data on finalised projects for this evaluation, there have been limitations in determining the (full) added value of the programme’s components.

From the available data and the collected information from beneficiaries and stakeholders, it appears that the (expected and yet seen) results delivered, could not have been possible without EU action and co-funding through grants or procurement. The extent to which the UAFP has enabled a more efficient use of financial resources for the Member States’ beneficiaries, than if Member States had acted alone, is, although not fully measurable at this stage, clearly identified by the consulted stakeholders and beneficiaries.

The following assessment is based on a thorough review of available reports and literature on the UAFP, targeted surveys of Hercule component beneficiaries and applicants, targeted surveys of AFIS users, and interviews with key stakeholders.

Interviews with beneficiaries and the case studies of selected projects showed that the supported actions focus on the output and benefits described below.

-    Improved investigative and surveillance capabilities and capacities by purchasing state-of-the-art communication surveillance equipment, upgrading existing surveillance systems and tools and training personnel to operate the newly acquired equipment;

-    Improved quality of collected data and evidence and the ability to exchange data with partners from other Member States and EU agencies and participate in trans-border investigations;

-    Improved performance of investigative and law enforcement authorities: (i) by purchasing equipment and software, which increases both the volume of data they can process and the speed of processing; and (ii) by enabling the use of artificial intelligence and digitisation in activities like evidence review, data classification and identity verification;

Training activities, conferences, study visits and publications have improved transnational cooperation among the parties involved in the protection of the EU’s financial interests and have contributed to applying a multidisciplinary approach in combating fraud and other illegal activities, including by bringing together academic researchers and practitioners.

Many respondents highlighted additional benefits beyond the scope and efficiency benefits mentioned above. One beneficiary of technical assistance highlighted that the UAFP programme was vital in ensuring that their office could stay up to date. This respondent stated that IT technologies develop very fast and are adopted very quickly by organised criminal groups. It is very difficult for the respondent to keep up with these developments, and they consider UAFP funding as a vital resource helping them stay up to date 136 .

Two beneficiaries of training activities highlighted that the participation of OLAF greatly improves interest and encourages participation in training and conferences. OLAF funding lends substantial credibility to the training activities, which results in more people attending them 137 .

Lastly, an applicant for training activities stated that funding for these activities at an EU level gives more added value because the assessment process is more neutral. At national level, there is a much smaller circle of professionals involved in this area, which makes an objective evaluation of the proposals difficult. By contrast, EU funding ensures a neutral review process 138 .

As for potential contributions to sustainable development goals and other benefits, the UAFP programme provides additional benefits partly through its contribution to meeting key sustainable development goals. Specifically, financial support from the EU through the technical assistance grants contributes to the sustainable development goal of reducing inequalities within and among administrations in Member States. The technical assistance grants support this goal by indirectly helping to harmonise the financial resources available to Member States by bolstering the budgetary capacity of national administrations 139 .

Furthermore, the programme indirectly helps build effective, accountable and inclusive institutions through its support to law enforcement agencies. This support is particularly evident through grants dedicated to supporting an improved digital transition 140 .

4.3Is the intervention still relevant?

Relevance addresses the relationship between the needs and problems at the time of the implementation of the intervention – in this case, over the 2021-2024 period of the UAFP. It also considers the relationship between current and future needs and problems in the EU in this policy area according to the objectives of the intervention.

The ex-ante evaluation of the UAFP in 2017-2018 identified the needs of stakeholders working within the programme (particularly at Member State level) in relation to the fight against fraud for the upcoming period. It concluded that support provided at the time by Hercule III was crucial to countering cross-border fraud against EU financial interests and that this should remain a priority for the EU, to be pursued by the next such iteration.

The main needs identified during the legislative preparatory works towards the adoption of the UAFP Regulation were:

-Lack of up-to date technical equipment and support in the Member States;

-Increase in new forms of fraud and rapid development of sophisticated technical tools used by criminals ;

-Need for cooperation between stakeholders (including the exchange of best practices and experiences);

-Need for access and exchange of data and information between stakeholders and Member States. 

The interim evaluation has revealed that the UAFP's specific objectives generally align with the programme's broader goals, showcasing its relevance overall. Nevertheless, there is room for improvement in providing targeted support to address emerging crime trends, particularly through the adoption of digitalisation measures.

According to stakeholder feedback, survey responses and interviews, the specific objectives of the Hercule component are strongly aligned with the UAFP’s general objectives and with stakeholder needs.

Overall, the UAFP is and remains relevant to emerging trends in fraud prevention by addressing digitalisation and e-commerce-related fraud through funding initiatives and training projects. While there is a growing emphasis on digital preparedness to combat evolving criminal tactics, efforts to address emerging trends explicitly should be consolidated.

The AFIS component's annual work plan, user satisfaction surveys, and functionality assessments demonstrate its relevance in facilitating information exchange and operational activities in customs matters.

For example, the AFIS component, particularly the Tobacco Seizure Management Application (ToSMA), contributes to combating tobacco smuggling by enhancing investigators' awareness and leveraging statistical models.

The IMS component, while perceived as useful and relevant, has, until very recently, faced challenges in effectively supporting the reporting of irregularities and combating fraud within the EU budget. Stakeholders pointed to the need for system upgrades, for example. This upgrade has now been launched in October 2024 and requires some time to get fully implemented this and next year, addressing most challenges identified in this evaluation.

The expansion of the programme’s beneficiaries to Ukraine and, potentially, other EU candidate countries since 2023, as well as increasing market prices for hardware and artificial intelligence (AI) technologies, could further limit the current (financial) ability of the programme to address forthcoming and increasing request for financial support to answer the needs.

For example, the Hercule component already rejects about two-thirds of all grant applicants each year since 2022, due to budgetary limits. This may suggest increasing and unsatisfied financing needs by the Member States and the associated country Ukraine, which could prompt frustration and dissatisfaction among applicants in the future.

Fluctuating inflation rates also have an impact on the budget that can be allocated to a project. As a comparison, in Belgium, the inflation rate in 2022 was 10.32%, up 7.11 percentage points from 3.21% in 2021.

In 2023, the inflation rate in Belgium was down again to 2.29%, a decrease of 0.92 percentage points between 2021 and 2023.

Figure 8 hereafter visualises the fluctuation in annual inflation rates in % for the EU-27 and within the euro area, triggering the rising trend in applications for financial support in the field of technical assistance funded by the programme to include a forecasted price increase in the planned application budget for a new project (e.g. between 5% and 10% added for equipment or IT tools as the procurement procedure can take up to one year to finalise).

In this sense, a reflection on a steady and substantial budget increase for the (future) programme may need to be considered.

Figure 8: Annual inflation rates in % for the EU and within the euro area (Eurostat) 141

5What are the conclusions and lessons learnt?

This section presents the conclusions and lessons learnt of this interim evaluation, based on the external study report and available data and documentation, consulted throughout this evaluation exercise.

5.1Conclusions

At this moment in time, passed halfway the programme’s implementation period, it can be stated that the UAFP has achieved its two general objectives, as well as its three specific objectives, so far and within the limits of the budgetary possibilities. The evaluation found that the specific objectives of the programme are generally relevant to the general objectives.

The activities implemented under the Hercule component have been found to be highly relevant to the specific objectives of the programme as well as to catering convincingly to the needs of applicants and beneficiaries of the programme.

The yearly calls for proposals, the various activities proposed within them, and the tools provided to users (AFIS in particular) are highly relevant to the needs of beneficiaries, participants, applicants and users of AFIS platforms.

The programme is structured in such a way that allows for beneficiaries to respond to fast-emerging trends in the fight against fraud and sets a promising starting point for such responses in the future.

Most projects carried out under the Hercule component of the UAFP could not have been completed if they were only funded at national or regional level. Even where national funding is available, without the support of EU funding, the projects carried out under the Hercule component could not be completed with the same scope and the same level of quality.

The research funding provided under UAFP occupies a unique position, and without this funding, many important research topics relating to the protection of the EU’s financial interests would simply not receive funding or sufficient funding.

The programme’s actual performance has matched the beneficiaries expectations in a large way and in different aspects, including unexpected benefits, both for programme beneficiaries as for the general public, for example by contributing to two (UN) sustainable development goals (‘SDG’), described hereafter. Although participating in the programme does require input from human and financial resources (minimum 20% co-financing for ‘Hercule’ projects), the UAFP has enabled national investigative units to optimise their use of resources, leading to a more effective fight against serious crimes that undermine the EU's financial interests.

On cost savings, the interim evaluation of the first component of the UAFP, Hercule, revealed that beneficiaries experienced some reduction in costs. On the application process, the current technical application process provided by OLAF is a substantial improvement compared with the previous programme’s process and is recognised as being highly efficient.

The increased internal coherence achieved with the three-component structure under the UAFP (compared to the Hercule III programme) is primarily observed in the reduction of the administrative work and costs, simplification in the management of funding, budget flexibility and redistribution of funds within and across the three components.

Nevertheless, there is room for further internal coordination (within OLAF), particularly in the early annual preparation and development of the AWPs and in the financial resource allocation, to ensure better coherence between the components of the programme and the priorities identified.

On the external coherence, the closest synergies and potential for complementarity is observed between the UAFP and the CCEI (managed by DG TAXUD).

As already mentioned, there are also common points between the UAFP, the Customs programme and the Fiscalis programme, both managed by DG TAXUD.

Although each year all the applications filed under the Hercule component require careful analysis and coordination between OLAF and DG TAXUD to avoid potential overlaps, no significant duplications were found. The relevant Commission departments will continue working closely at different levels and at different stages in the grant process to ensure it is clear to applicants and beneficiaries how the programmes work and what activities are eligible for funding.

Overall, EU financial support provides a substantial benefit to the scope and efficiency of projects funded through the programme. The annual overview and annual programme performance reporting indicate that the structure of the programme may support key synergies, such as the flexibility to transfer funding between and within components.

The UAFP programme provides additional added value through reputational gains. Receiving UAFP funds and participating in OLAF training and networking projects result in a reported increase in the interest, credibility and level of participation.

The programme implementation has shown the existence of additional benefits to the EU citizens, in particular through the programme’s indirect contribution to meeting key sustainable development goals (SDGs). Specifically, financial support from the EU through the technical assistance grants contributes also to the sustainable development goal specifically linked with reducing inequalities within and among countries (SDG Number 10), here at national and/or regional governance level in particular.

The technical assistance grants support this goal by indirectly helping to harmonise the financial resources available to Member States by bolstering national administrations’ budgets.

Furthermore, the programme indirectly helps build effective, accountable and inclusive institutions in the beneficiary countries by supporting law enforcement agencies, particularly through grants to help fund an improved digital transition (SDG Number 16).

5.2Lessons learnt

This evaluation and the study have made it possible to learn about the existence of, for example, unexpected benefits for the beneficiaries of the ‘Hercule’ component from the successful implementation of the programme so far.

Also, a number of challenges remain to be tackled further as described hereafter.

Many consulted respondents highlighted additional benefits beyond the scope and efficiency benefits mentioned earlier. The respondents consider UAFP funding as a vital resource helping them stay up to date with the latest technologies in their fight against fraud and other offences detrimental to the EU’s financial interests.

The programme’s funding also lends substantial credibility to (the organisers of) the training activities, which results, among other benefits, in more participants and better networking.

This reinforces the idea that the level and scope of funding under UAFP should at least be maintained as applicant and beneficiary feedback shows that the programme fills a unique niche. Any reduction in funding opportunities through UAFP would leave a significant gap in the fight against fraud or any other irregularities detrimental to the EU’s financial interests.

This interim evaluation noted that the number of (mainly) technical assistance proposals for the Hercule component of the UAFP has doubled in recent years (between 2022, 2023 and 2024). This significant increase has shown that the Commission’s (OLAF’s) efforts, in 2022 and 2023, to organise workshops for beneficiaries has been boosting the success of the programme.

Informing potential beneficiaries of the calls for proposals in a more targeted way could further increase the number of applicants and the number of Member States participating in the programme, especially from countries that submit none or fewer applications.

Targeted information sessions involving Member State institutions could be organised in countries where the number of EU fraud cases and irregularities is high but few applications are submitted. This may help increase the participation rate and result in a more even distribution of the funds among as many Member States as possible, including the associated partner Ukraïne. Nevertheless, some countries might not be interested in participating in the UAFP.

Transnational cooperation can be improved by providing incentives to beneficiaries to set up multinational project teams and multi-beneficiary projects.

The (‘Hercule’) project reporting platform should be upgraded to make it more user-friendly and intuitive for beneficiaries.

A more quantitative comparison of costs and benefits of the programme could be facilitated with data on benefits. This would allow for these benefits to be monetised as far as possible, e.g. more standardised reporting on the monetary value or estimated value of benefits enjoyed by Hercule grant recipients.

The application procedure for Hercule grants has significantly improved, compared with the previous edition and is considered to be highly efficient. Nonetheless, there is still potential for improvements in efficiency. The efficiency of the application process may benefit from including, if technically possible, a feature in the Commission’s online portal for calls and tenders, to notify applicants (directly and/or automatically) when a call has been launched.

Greater user-friendliness for users with disabilities, such as those with visual impairments, may also make the process less resource-intensive for some applicants.

The evolving challenges in the customs anti-fraud environment call for a reinforcement of the financial means of the activities under Council Regulation No 515/97, in particular for the modernisation of the AFIS infrastructure and the enhancement of the data management and data exploitation functionalities. This would increase the operational capacity of investigators and data analysts to pro-actively detect cases of fraud and to monitor existing and evolving trends.

This modernisation process should be carried out with the use of the latest technologies and advanced analytical tools and techniques for searching and combining data, as well as visualising results. Machine learning, artificial intelligence (AI), text mining and advanced statistical techniques will provide an automated framework for processing big data (for example e-commerce data).

With regard to the further implementation of AFIS, it seems that, although the AFIS training and information materials are deemed useful, not all users are aware that they exist.

The increasing costs of the maintenance and development of the AFIS platform and the elevated costs of the needed state of the art technology and equipment for its modernisation call for a significant increase in the allocated budget within the (future) programme.

The databases under the AFIS umbrella are all considered efficient. Resource utilisation can be optimised and less hardware is required on a central EU level. Administration costs are lower. As data is stored centrally, fewer components are needed and the data storage is done in a coordinated way.

The existing services offered by AFIS as an intelligence platform to facilitate the sharing of the results of analysis carried out at EU level sould be further enhanced. Certain AFIS modules show exceptionally strong evidence of benefits (e.g. CIS+).

Current trends in crime show that projects enabling the fight against digitally enabled fraud require more funding. Hence, the future overall programme budget might benefit from a planned increase to address and mitigate the increasing proportion of applications in the Hercule component of the programme that are unsuccessful, mainly due to budgetary limits (now about three quarters of all the applications under technical assistance).

Applicants highlighted the lack of similar financial support from national or other sources.

The high costs of hardware and the need for policymakers to keep up with technological developments might require an increase in the budget of the (future) programme to continue supporting national authorities in combating fraud, corruption or any other illegal activities.

Lastly, such a budget increase should take into consideration the extension of the programme’s geographical scope, following Ukraine’s association to the programme since 1 January 2023.

In this way, the potential is created for extending the programme to other candidate countries.

-Lead DG: DG OLAF

-DECIDE Reference: PLAN/2023/753

-The specific Inter-service Steering Group (ISG, composed of staff members from DG SG, SJ, TAXUD, JUST, HOME and BUDG) agreed with a specific Call for Evidence with targeted questions/interviews to/with dedicated groups of interest as the stakeholders of the programme are very specific national or regional services dealing with the protection of the EU’s financial interests, among other priorities. The general public has not been specifically consulted. The ISG has held three meetings, one for the approval of the Terms of Reference for the selection of the contractor, one for the presentation of the inception report by the contractor ICF and one for the presentation of the draft final study report by ICF (on 16 April 2024).

-The interim evaluation preparatory activities were launched in the autumn of 2022, including the reopening of a competition within an existing Commission Framework Contract (FWC) from DG HOME between the companies within that FWC. This in order to select one company to set up and finalise an external study in order to assist the Commission (OLAF) with this interim evaluation exercise. The contract was signed with the Belgian branch of the company ICF, located in Brussels, together with its subcontractors. The contract ran between 17 May 2023 and 17 May 2024, resulting in the submission of a final study report 142 on the interim evaluation on 17 May (last update 9 July 2024). The Report from the Commission with its accompanying staff working document is due to be adopted by 31 December 2024 at the latest. The study and the Commission Report (+SWD) will be published on the Commission (OLAF) website. The ICF study report (finalised in July 2024 for publication) is available here: https://data.europa.eu/doi/10.2784/1075235 .

1.Principles

The external study by ICF on the interim evaluation of the programme, including four extensively developed case studies, is considered to be of high quality (taking into account the rather limited number of finalised projects and the low response rate of the surveys) and has respected and followed the following principles, as required by the Better Regulation’s guidelines on evaluations:

§comprehensiveness – covering the following (five) criteria:

oeffectiveness;

oefficiency (including sustainability or durability of the action/materials);

ocoherence (both internal and external);

orelevance; and,

oEU added value.

§proportionality – the scope of the evaluation (study) has been tailored to the particular intervention, the time since its implementation and the available data.

For some criteria, new data had to be collected, analysed and compared with other findings. For others, a short summary has been provided, on the basis of existing reports and information, or a standard explanation has been provided.

§independence and objectivity – Through ICF, an independent and objective evaluation has delivered robust and reliable results. The analysis done in the evaluation is considered to be independent and objective, including for the staff working document and the Report from the Commission based on that study, since it has been based on all relevant information; it has been conducted without influence or pressure by third parties and the study and the SWD and Report reflect transparently the positive as well as the negative elements of the analysis;

§evidence-based approach – This evaluation has been based on the best available evidence drawn from a diverse and appropriate range of methods and sources (triangulation). Not all sources were equally robust considering when and how the (reduced) available evidence was collected. As far as could be identified, the evidence is considered not to have been subject to bias or uncertainty.

Any limitations related to the evidence and the methodology, particularly in terms of their ability to support the conclusions, has been clearly explained, both in the study and in the introductory part of this staff working document.

-ICF has been able to establish a clear link between the evaluation questions addressed and the corresponding methodology;

-The main evaluation questions which have been addressed, in line with the Better Regulation Guidelines’ evaluation criteria (mentioned above) are:

§What have been the outcomes of the interventions, implemented by the programme, so far, considering the ex-ante evaluation of the legislative proposal in 2018, the preamble of the adopted legal text, and the final evaluation of the Hercule III programme in 2020?

§How has the programme implementation evolved over the interim evaluation period (considering specific circumstances such as e.g. the impact of Corona measures in the Member States)?

§To what extent can the intervention be considered successful and why? Where possible, each of the five main evaluation criteria have been assessed: effectiveness, efficiency, coherence, relevance as well as EU added value of the programme’s actions, also in relation to other programmes (synergies, complementarity) with similar objectives and/or target audience (in particular, the CCEI and the Customs programme).

§Are the objectives and the activities of the intervention still relevant today and for the future?

§How did the EU intervention make a difference, compared to what has been (financially or administratively) possible on a national or regional governance level?

§What is the EU added value of the programme?

§What are the conclusions and lessons learnt, as well as the way forward?

-Quantitative and qualitative indicators have been identified by ICF and used as much as possible throughout the evaluation process;

A representative sample of measures and activities examined has been drawn up (and agreed upon, in consultation with the contractor and the ISG, the inter-service steering group, at the presentation of the Inception Report) in a manner suitable for each evaluation question, and has enabled the evaluators to draw general conclusions on the measures and address the key application issues, as identified by the inter-service steering group and the consulted stakeholders.

The sections hereafter present the methodology used in the ICF study to support the interim evaluation of the UAFP. It utilises a mixed-methods approach, comprising both qualitative and quantitative research techniques, to answer the evaluation questions shown in the evaluation framework. Figure 9 below presents an overview of the methodological approach.

Figure 1: Schematic overview of the methodological framework

Source: ICF analysis.

2.Study tasks

Source: ICF analysis.

3.Data collection tools 

To gather the evidence necessary to evaluate the UAFP, the study team used several data collection tools:

§desk research;

§documentation analysis;

§stakeholder consultations;

§case studies 143 .

4.Desk research 

The desk research comprised the collection and analysis of secondary sources (included those provided by OLAF) for the period of the interim evaluation:

§policy, legal, and strategic documents, such as policy and legal documents at EU and national level;

§programme-specific documents, including PIF reports and (AFIS) steering committee meeting inputs;

§action-level documents, including grant applications, implementation reports, and final reports of grant recipients;

§other relevant documents, including academic or other (external) studies.

ICF collected, catalogued and systematically stored the relevant sources, which were then coded and collated into an evidence table. The data gathered during the desk research were analysed and incorporated into the answers to the evaluation questions (see section 5).

5.Documentation analysis

The documentation analysis consisted of a detailed review and analysis of key programme documents for each component of the UAFP:

§the AFIS surveys in 2019, 2021 and 2023, Summary reports;

§OLAF’s annual activities reports;

§the 2021 study report on the final evaluation of the Hercule III programme;

§PIF reports on fraud, covering the years 2017-2023;

§annual overviews with information on the results of the UAFP in 2021, 2022 and 2023;

§progress reports and final reports (where available) for projects funded under the ‘Hercule’ component of the UAFP;

§grant agreements for funding awarded under the UAFP;

§data on (pending) applications for funding under the UAFP, provided by OLAF;

§minutes of the AFIS steering committee meeting in December 2022, on the protection of the European Union’s financial interests;

§documents and presentations from the AFIS steering committee, covering programme implementation and budgeting.

The contents of these documents were analysed and incorporated into the findings.

6.Stakeholder consultations

The stakeholder consultations included three targeted online surveys, targeted interviews, and a planned workshop. The results of the stakeholder consultations can be found in Annex V.

The study foresaw targeted surveys to gather inputs from national-level stakeholders. They included surveys targeting applicants to the Hercule component call for proposals, including beneficiaries, applicants awaiting a decision, and applicants whose application had been rejected, as well as a survey of registered AFIS users.

ICF completed and sent surveys to all applicants to the Hercule component, which participated in the calls between 2021 and 2023. To maximise the response rate, ICF extended the survey period and followed-up with survey recipients to encourage a response. All participants received the survey twice and, in cases where the survey was not complete, they also received individual reminders/invites to complete the survey.

This round of surveys produced 29 usable survey answers from beneficiaries (29 successful respondents) and 14 usable survey answers from applicants (14 usable respondents). Only one unsuccessful participant responded. Survey respondents provided insights into the process and results of their applications and interventions (where successful) under the UAFP.

ICF analysed the responses and incorporated the results into the findings of the study.

The survey of the Council Working Party on Combating Fraud was drafted by ICF and distributed by the Council Secretariat-General through OLAF. It yielded just three usable replies.

The targeted survey of AFIS users was completed, with ICF providing inputs and survey questions, and circulated directly to AFIS users via AFIS Liaison Officers in the Member States. ICF received the raw data from the surveys on 9 February 2024.

§The study team conducted eight scoping interviews at the inception stage, as well as further scoping interviews with OLAF officials during the interim phase of the study to understand how AFIS functions and to adjust the additional interview questionnaire for the AFIS satisfaction survey;

§The study team conducted 43 stakeholder interviews at national and EU level. The results of the interim report were used to identify data gaps and adjust the interview questionnaires to address those information gaps. The study team consulted the following stakeholders:

oBeneficiaries of the Hercule component (technical assistance and training activities);

oEuropean Commission stakeholders (particularly on coherence): staff of various Commission DGs;

oAFIS and IMS stakeholders.

§Finally, ICF incorporated inputs from a stakeholder workshop on 16 April 2024. It presented the draft final report to European Commission officials (Inter-Service Steering Group members and other Commission staff) at OLAF’s premises.

7.Case studies

The purpose of the case studies 144 was to thoroughly examine the main themes of the evaluation. The Regulation introduced notable themes and innovations in respect of the primary focus and structure of the UAFP. As the programme is still being implemented and many actions are ongoing, the case studies focused on assessing progress towards achieving the strategic direction for the UAFP, rather than evaluating longer-term impacts and outcomes (typically done in final ex-post evaluations).

By taking this mid-term perspective on their design, the case studies offer a comprehensive understanding of whether the programme is advancing towards its objectives, allowing for adjustments or re-orientation if necessary for the remainder of the programme. The study team completed four case studies. The case studies covered the following topics:

1.Case study on investigative capacity;

2.Case study on digitalisation, AI, data analytics;

3.Case study on transnational cooperation and exchange of information;

4.Case study on training related activities.

The results of the case studies have been incorporated into this report. Full details of the four case studies are available in Annex 3 of the study report, available here: https://data.europa.eu/doi/10.2784/1075235 .

8.Limitations to methodological approach and robustness of findings 

This section outlines the primary constraints and methodological adaptations encountered during the study.

Given the delayed adoption of the UAFP’s legal basis and the late implementation of the 2021 activities (COVID-19), the completion of project outputs and subsequent reporting processes has been held up.

Consequently, drawing conclusions on the programme's long-term impacts and outcomes proves challenging. In addition, quantifying the extent to which the UAFP has met its overarching objective and assessing its long-term effects are hindered by the indirect nature of UAFP-funded activities. For instance, establishing a direct causal link between the programme's actions and their impact on the financial interests of the EU, such as through the training of national customs authorities, presents difficulties.

To address these issues, the study team took several measures during the inception phase:

§Streamlined the impact indicators and questions within the analytical framework, recognising the anticipated scarcity of relevant data for comprehensive analysis;

§Revised the analytical framework and associated data collection tools to prioritise qualitative indicators and data;

§Engaged stakeholders to provide insights into the expected achievement of the UAFP’s objectives, facilitating qualitative assessment of progress. This involved soliciting stakeholder feedback on project alignment with objectives, including for projects still in progress.

These adjustments enhanced the study's adaptability to the dynamic nature of UAFP implementation and its indirect impact pathways.

1.The evaluation matrix 

The evaluation framework, presented hereafter in a schematic format, has been developed following the key principles of the European Commission’s Better Regulation Guidelines and its toolboxes and evaluates the relevance, efficiency, effectiveness, coherence, and added value of the UAFP.

Table 1 – Evaluation matrix