Français (FR)English (UK)

History and presentation

The working group on program evaluation was created in 1992 at the 14th INTOSAI Congress in Washington. One of the themes of the Congress was “Program Evaluation: An Emerging Approach.” The debates were conducted on the basis of a general report presented by France and led to the establishment of this working group that kept working ever since.

By 1995, at the 15th INTOSAI Congress in Cairo, the Working Group on Program Evaluation had “generated considerable discussion among many delegates who,” as the International Journal of Government Auditing reported, were “just beginning to work in this relatively new area” ("15th INTOSAI Congress held  in Cairo", International Journal of Government Auditing, January 1995). The result was that the Congress approved its work and recommended, among other things, that it

“elaborate(s) a methodological framework that would facilitate evaluation work, including an overview of evaluation concepts, objectives and standards. This document should examine what changes are suitable in the organization and working methods of SAIs in order to allow them to implement evaluations (use of external experts, incorporation of research disciplines, methodological steering committee; and reporting).” ("15th INTOSAI Congress held in Cairo”).

Accordingly, at the 16th Congress in Seoul in 1998, the Working Group presented a draft report on the methods and practices of evaluation, intending to present the final product for adoption at the 18th Congress in Budapest in 2004 (« Cooperation produces results », International Journal of Government Auditing, January 2002).  As announced, this text was presented to the 142 SAIs which attended:

“The working group, chaired by the French Cour des comptes, presented a draft document, Program Evaluation for SAIs: a Primer, at the Budapest congress. This document summarizes basic program evaluation concepts and includes examples from SAIs and professional evaluation associations and groups from around the world. It discusses the interrelationship between performance audit and program evaluation, a definition of program evaluation and related technical terms, challenges in planning and designing evaluations, program evaluation methodology, new ways of working with program evaluation and future prospects for program evaluation.” ("The INTOSAI committees and working groups present their upcoming reports and work agendas", International Journal of Government Auditing, January 2005). 

The working group has thus established a link between program evaluation and performance auditing, defined in the INTOSAI Implementation Guidelines for Performance Auditing. As stated in the following quotation, the definitions of each concept and their differentiation are not always clear: 

“The present debate about the role of the performance audit and its methods … is sometimes characterized by uncertainties in definitions of concepts and in confusion over objectives and methods in an expenditure program. When working on a performance audit, it is essential to establish and make known that the scope of the audit is limited to verifying the degree to which the objectives of a program have been met, and to analyzing the means that have been used to reach these objectives. The audit never questions the objectives. In an evaluation situation, you may choose also to question the objectives as part of the study.” (Inga-Britt Ahlenius, “Performance Audits, Evaluations and Supreme Audit Institutions”, International Journal of Government Auditing, January 2000). 

The 19th INTOSAI Congress held in Mexico City gave the opportunity to place this reflection as well as the work of the working group within the framework of INTOSAI’s strategic objectives and, more particularly, in the context of its 1st strategic goal focused on the implementation of professional standards, currently under construction. A large portion of the auditing activities of SAIs lends itself to the adoption of professional standards. However, these standards are more restrictive with regard to financial or compliance audits compared with performance audits, where auditors have to show initiative and be quick off the mark. As for evaluation, even if there were clear links with performance audits, the methods and tools used have to be adapted to its specific purpose and each time the approach used is original. Accordingly, evaluation is not a matter for standards, but rather is founded on a set of methods specific to SAIs or taken from other fields such as social sciences. Thus, evaluation clearly falls within the scope of INTOSAI 3rd strategic goal, devoted to knowledge sharing and best practices. 

The report of the working group Program evaluation for SAIs: introduction is thus fully in line with INTOSAI’s strategic plan for 2005-2010, which states that: 

“Communication, cooperation and collaboration have been hallmarks of INTOSAI since its inception in 1953 (…) Specialized committees and working groups have been established to research, develop and publish methodologies, guidelines and best practices for SAIs in various disciplines, such as (…) program evaluation... INTOSAI’s seven regional working groups are essential components of the organization, offering their members many services and publications designed to share knowledge and facilitate collaboration within each region (…) INTOSAI proposes to expand the benefits of the regional working groups by facilitating the additional sharing of the key work, successes, and lessons learned for individual regions with all other regional working groups. In this way, information would be shared horizontally, or globally.” (INTOSAI, Strategic Program 2005-2010, Vienna, Austria, 2005, p. 14). 

The following years were devoted to finalizing the text of the report, which was presented in the five official languages during the 20th INTOSAI Congress in Johannesburg. The meeting of the working group held in Paris in May 2009 allowed a framework for action to be mapped out for the coming period: in accordance with the guidance in INTOSAI’s strategic plan for 2011-2016, the working group fixed two objectives: to promote the exchange of experiences among SAIs and to contribute to the collection of case studies and best practices.

The program evaluation working group wishes to thank the US Government Accountability Office for its decisive contribution in the drafting of the report Program Evaluation for SAIs: introduction. It would also like to thank those countries which were kind enough to comment on the draft report and its successive versions. These comments were taken into account, where relevant. All internet links included in the preliminary version were updated and checked upon at the time this document was drafted.