A joint project between the National Audit Chamber of South Sudan (NAC), the INTOSAI Development Initiative (IDI) and AFROSAI-E between 2017 and early 2020 has now been evaluated. According to the evaluation, the peers from IDI, AFROSAI-E, SAI Kenya and SAI Norway have provided excellent support and continued support from the present peers would be of great value. Still, the evaluation points to several areas of improvement and gaps in the overall goal achievement.
The overall objective of the project was to “Maintain and strengthen key audit-related capacities in NAC, to prepare NAC to play a strong role in the reform efforts of the Government of South Sudan to improve and sustain Public Financial Management Administration and accounting systems.” There were six expected outcomes of the project:
- A relevant and enhanced regularity audit function in NAC
- A relevant and enhanced performance audit function in NAC
- Core audit management and HR systems in place in NAC
- Key stakeholders are familiar with NAC’s function, audit findings and how reports can be utilized
- NAC is developing in line with the ISSAIs and international best practices
- NAC's capacity development is strategically managed and well-coordinated
The evaluation was carried out by AB Professional and noted that during the project period, the NAC was highly challenged especially because the Government of South Sudan (GOSS) did not meet its funding commitments. For example, NAC staff did not receive their salaries on time and as at March 2020, they had not been paid for four months. In addition, funding for audit operations has not been adequate and hence staff have difficulties in getting transport to work, transport to auditees, there is limited access to the Internet, long periods of no electricity, no Internet at the NAC, occasionally no water in the office or toilets, and no adequate working environment and infrastructure in general.
The evaluation further noted that no audit reports from the NAC have been tabled in the Parliament since 2013 and even though audit reports are submitted to the President’s Office, the NAC is not invited to present these reports in Parliament, where they would be public documents. As such, the basic conditions for playing the role of a SAI are missing. IDI has therefore classified this project as ‘lifeline support’, which has the objective of maintaining the basic competency and skills of the SAI and its staff in the interim period till the situation stabilises and more long-term support can be provided.
Despite the label ‘lifeline support’, the Project had an ambitious Results framework. Of the planned audit reports for completion, a number were brought to printing stage by the time of the evaluation (they were printed afterwards). Other key deliverables were also achieved for example, the NAC prepared a Strategic Plan for 2019-2024 and a Stakeholder Engagement Strategy, the generic AFROSAI-E performance audit manual and human resource development manual were customised for on-the-job training and future use in NAC, the Human Resource Development Strategy was prepared, the NAC annual performance report covering 2005-2018 was completed and issued, staff undertook training courses with AFROSAI-E. and key auditees and PAC members were sensitised on NAC functions and the handling of its reports. However, the risks were underestimated in the project description and as such, so many of the targets have not been fully achieved. Despite the few delayed deliverables, the results are so far positive considering the challenging situation within which the NAC operates.
On the overall goal achievement, the evaluation had four conclusions; (1) the two indicators of goal achievement were not appropriate for measuring the overall success of the project, (2) measurement has not been done in accordance with the definitions, (3) figures that are used in the reporting are of questionable reliability, and (4) the overall achievement of the formal goals is low, but mainly attributable to causes outside the project management’s area of influence.
The evaluator further pointed out that there has been no attention to advancing gender equity in the NAC since no indicator on gender was included in the Cooperation Agreement nor in the Results Framework in the Grant Agreement between the Ministry of Foreign Affairs of Norway (MFA) and the IDI. The NAC has developed a new Strategic Plan (SP) 2019-2024, which also makes no reference to gender issues. According to the evaluator, the SP should be reviewed based on a proper gender analysis and an Addendum added. The importance of gender equality should be advocated with emphasis and the way forward clearly spelt out in strategic and operational plans.
Furthermore, the evaluator noted that it was relevant to provide ‘lifeline support’ during 2017-2019. However, if the NAC is not provided with the basic resources for fulfilling its role as SAI by the government, it will be unable to complete outstanding audits and the drain of trained auditors will continue. While the future role of the NAC and its obligation to report directly to the legislature is under high-level discussion, it can continue to issue its reports to the Executive authorities and start building demand for accountability through other avenues such as civil society organisations. Thus, the proposed second phase of the Peer-support project – the “NAC Strategic Change project 2020-2024”[1] should be conditional on full and continuous funding from the GOSS for all staff, adequate office space and transport as well as other operational costs. This condition is vital if the proposed project is to be successful and the project should be terminated on three months’ notice if the condition is not met.
For future support, the evaluator concluded that the IDI is the actor that is best positioned to be responsible for the support in the next phase. The peers from IDI, AFROSAI-E, SAI Kenya and SAI Norway have provided excellent support and continued support from the present peers would be of great value. However, auditors are not experts in all areas where support is needed and thus, adequate expertise should be engaged for (among others) organisational development and development of internal and external processes.
Monitoring and evaluation are important elements in the project cycle and provide reflection on the performance of the project. Evaluation enables the project to receive independent feedback on the relevance, effectiveness, efficiency and/or consistency of the project. Monitoring the Peer-support project has been a challenge, and hence in the proposed second phase of the project, consideration should be made to include an ongoing evaluation[2], that continuously provides feedback to the project management and facilitates the learning process.
The project is presented in more detail here in the final project report. This report also includes the action plan for each of the recommendations of the evaluation.
[1] NAC Strategic Change Project 2020-2024, Cooperation Agreement, signed 23 August 2019
[2] Ongoing evaluation is one of the three state of the art types of evaluation (the other two are ex ante and ex post evaluations). It means that the evaluation is ongoing during the project period. Ongoing evaluation is “a source of feedback, a tool for improving performance, an early warning of problems (and solutions) and a way of systematising knowledge” (European Commission: Practical Handbook for Ongoing Evaluation). In conjunction with evaluation information, effective monitoring and reporting provide decision-makers and stakeholders with the knowledge they need to identify whether the implementation and outcomes of the project are unfolding as expected and to manage the initiative on an ongoing basis. Especially when there are external risks that are significant threats to achieving the goals it is quite common to include an ongoing evaluation as a component of a project. An alternative is a Mid-Term Evaluation, which can be implemented at any time.
 |
For more information on IDI's work in South Sudan, click here. |