Audit of the Canadian Space Agency Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology

Audit Report

Project No. 12/13 01-01

Prepared by the
Audit and Evaluation Directorate

May 2013

1.0 Summary

1.1 Audit objective

The objective of the audit project was to determine whether a management framework is in place to ensure that the Canadian Space Agency (CSA) Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology (the Program) is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and whether there is proper accountability reporting of the Program.

1.2 Audit opinion

In my view, the management framework for the CSA Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology has certain deficiencies that require management's attention. However, accountability for the Program's operations was deemed appropriate.

1.3 Statement of assurance

It is my opinion, as Chief Audit Executive, that sufficient and appropriate audit procedures have been followed and evidence gathered to support the accuracy of the opinion contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time of the audit, against pre-established audit criteria determined in consultation with the directorates concerned, and is only applicable to the particular entity being audited. The evidence was gathered in compliance with Treasury Board (TB) policy, directives and standards for internal audit. The procedures followed comply with the professional standards of the Institute of Internal Auditors. The evidence gathered is sufficient to convince senior management of the validity of the opinion derived from the internal audit.

1.4 Summary of recommendations

During our audit, it was determined that the CSA Grants and Contributions (G&C) Centre of Expertise had put in place a control framework and introduced best practices for the management of agreements. In general, we found that existing procedures had been followed to ensure compliance with Program terms and conditions and TB policies and guidelines regarding financial management and transfer payments. Furthermore, the Program's performance is subject to accountability reporting and results are measured.

Nonetheless, we found some deficiencies that require management's attention and for which we make the following recommendations:

  1. Apply existing controls systematically to ensure that files are documented in accordance with prescribed standards and procedures;
  2. Apply existing controls systematically to ensure that all relevant clauses and required project information are included in funding agreements, in accordance with the Directive on Transfer Payments;
  3. The internal directive concerning the auditing of recipients should be completed and an audit plan should be drafted and implemented.

The original version signed by the
Chief Audit Executive

Audit Team members

  • Dany Fortin, Senior Auditor
  • Louis Martel, Senior Auditor

2.0 Audit Report

2.1 Background and risk

In September 2002, the Government of Canada approved the creation of the Class Grant and Contribution Program to Support Awareness, Research and Learning ("the 2002 Program"). The objective of the Program was to support the CSA's priority objectives in order to allow the CSA to work closely with Canadian granting agencies, universities and industries with a view to advancing space knowledge, developing and demonstrating new technologies and assisting in the training of the qualified personnel needed by universities and the high-tech sector in Canada. In 2004, a new component was added to the Program. Program changes were approved by the Treasury Board (TB) in 2005 and 2007. In addition, the Minister exercised his authority, as prescribed in the Policy on Transfer Payments, to extend the terms and conditions of the Program until the end of March 2010.

An internal audit of the 2002 Program was conducted in 2005. The findings of this audit revealed that the majority of the sectors kept administrative records to ensure a satisfactory audit trail. Nevertheless, program managers reviewed some aspects that had been identified as requiring improvement. The two main recommendations of the audit were to improve performance measurement as well as the method used to determine recipient eligibility (including ongoing eligibility in the case of multi-year agreements).

In October 2009, the TB approved the new terms and conditions of the Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology ("the Program").

The Program initially had two components: research, and awareness and learning. Currently, only the research component is operational. The main objectives of the research component are to:

  • support the development of science and technology relevant to the priorities of the Canadian Space Agency;
  • foster the continuing development of a critical mass of researchers and highly qualified people in Canada in fields relevant to the priorities of the Canadian Space Agency;
  • support information gathering, studies and research related to space; and
  • support the operations of organizations dedicated to space research.

Budget for the CSA Class Grant and Contribution Program

2012-2013
$M
2013-2014
$M
2014-2015
$M
Grants 6.1 6.4 6.5
Contributions 1.5 2.1 1.3
Total 7.6 8.5 7.8

*Source: 2013–2014 Report on Plans and Priorities

The Program is subject to its own terms of reference, as well as to the TB Policy on Transfer Payments and related directives, and the federal Financial Administration Act.

The CSA's Grants and Contributions (G&C) Centre of Expertise is responsible for establishing the control framework for the management of G&C agreements, and providing the tools and training required for its application. The mandate of the G&C Centre of Expertise, as approved by the CSA Executive Committee, is to offer expertise with respect to G&C, and to provide management, supervisory and performance monitoring services for all of the CSA's G&C programs. The G&C Centre of Expertise also supports the activities of the G&C Advisory Committee.

G&C governance within the CSA includes a G&C Advisory Committee and a G&C Committee. The mandate of the G&C Advisory Committee is to act as a liaison with respect to the review and analysis of all activities and documents associated with the management of grants and contributions prior to the formulation of recommendations to the G&C Committee. The mandate of the G&C Committee is to serve as a forum for CSA senior management and to provide strategic and organizational direction for the selection, design, implementation and establishment of policies and departmental frameworks for the CSA's transfer payment programs. The Executive Committee acts as the G&C Committee.

Agreements are managed by the managers responsible for them within the various branches. As part of its mandate, the G&C Centre of Expertise works closely with and provides support for the managers of CSA G&C projects. The Centre may also manage initiatives and agreements under an MOU with the responsible branches.

This audit project is part of the 2012–2015 Risk-Based Audit Plan (RBAP) approved by the Audit Committee. During the drafting of the RBAP, the entity was evaluated based on its risks and its relative importance. It obtained a score of 6 out of a maximum of 16, which corresponds to a medium level of risk. The principal risk factors identified were the development of a new program, the establishment of a centre of expertise, the review of the control framework, the development of a computerized management tool and the time elapsed since the previous audit.

These risks were discussed with the auditee's representatives who were present at the audit launch meeting.

2.2 Audit objective, scope and approach

Objective

The objective of the audit project was to determine whether a management framework is in place to ensure that the Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and whether there is proper accountability reporting of the Program.

Scope

The audit project focused on expenses incurred under the Program from the time of its approval by the TB in October 2009 up until March 31, 2012.

Approach

The audit involved various audit procedures, including interviews with staff, reviews of policies, directives and procedures, and document analyses. It should be noted that no recipients were interviewed.

The audit criteria were based on legislation, policies, standards and/or best practices, and were discussed with management.

2.3 Findings, recommendations and management's response

To ensure that the Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, we expected to find that

  • further to the audit by the Office of the Comptroller General (OCG), the applicable recommendations had been implemented;
  • the procedures required for ensuring that G&C agreements between the CSA and recipients comply with Program terms and conditions had been followed;
  • the G&C Program is carried out in accordance with TB policies and directives on financial management and transfer payments; and that
  • there is accountability reporting of the auditee's operations, and that results are measured.

The audit objective and criteria were discussed with management.

2.3.1 Follow-up on the OCG's recommendations

Audit objective: A management framework is in place to ensure that the Class Grant and Contribution Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and that there is proper accountability reporting of the Program.

Findings
Criterion 1

With reference to the OCG's horizontal audit of the G&C management framework set up following an internal self assessment, the management action plan is appropriate and being implemented satisfactorily.

Condition

Implementation of an action plan following the OCG audit

In our view, the action plan implemented by management following the OCG audit is appropriate, and the plan's implementation is satisfactory.

Following the OCG's horizontal audit of the G&C management framework, published in 2012, the CSA G&C Centre of Expertise conducted a self-assessment in order to evaluate the CSA's position regarding the OCG's recommendations. An action plan was subsequently drawn up in response to the applicable recommendations.

These recommendations were as follows:

  • Strengthen roles and responsibilities with respect to G&C throughout the organization;
  • Design the tools needed to gather and analyze, on an ongoing basis, performance data relative to G&C programs;
  • Establish recipient monitoring activities and reporting requirements based on risk assessment strategies.

The following were implemented by management over the past year:

  • Development of a control framework and the tools required to standardize processes;
  • Implementation of a recipient risk management framework and related mitigation measures;
  • Development of phase 1 training tools;
  • Review of the Program's performance measurement strategy and data gathering tools.

The following action plan items have yet to be implemented:

  • Updating and communication of the Centre of Expertise mandate following the decision to broaden roles and responsibilities;
  • Development of phase 2 training tools;
  • Implementation of a recipient audit strategy;
  • Completion of the posting online of a recipient dialogue box.

The G&C Centre of Expertise has worked hard over the past few years to complete the development of the control framework required to manage G&C agreements. Thus, the implementation of the management action plan in response to the OCG audit completes the planned improvements to the control framework. The action plan is scheduled to be fully implemented by March 31, 2014.

Cause

N/A

Effect

N/A

Recommendation

N/A

Identified Responsibility
Organization

N/A

Function

N/A

Management's Response

N/A

Management Action Plan
Details of the action plan

N/A

Deadline: N/A

2.3.2 Compliance with the terms and conditions of the Program

Audit objective: A management framework is in place to ensure that the Class Grant and Contribution Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and that there is proper accountability reporting of the Program.

Findings
Criterion 2

Required procedures were followed to ensure that G&C agreements between the CSA and recipients comply with Program terms and conditions.

Condition

Compliance with Program terms and conditions

In our view, procedures for ensuring that G&C agreements between the CSA and recipients comply with Program terms and conditions were followed in the majority of cases. However, we identified some aspects that require management's attention and for which we have made a recommendation.

For the purposes of this criterion, a sample of 10 agreements totalling $6.7 million was selected and reviewed. All of the agreements included at least one disbursement made between September 1, 2009, and March 31, 2012. The sample had the following characteristics:

  • 8 grant agreements and 2 contribution agreements;
  • 1 agreement for an amount greater than $3 million, 2 agreements for an amount greater than $800,000, 2 agreements for an amount greater than $400,000, 4 agreements for an amount greater than $100,000 and 1 agreement for an amount less than $5,000;
  • Almost all of the agreements were entered into with Canadian universities.

The files were selected based on the auditors' judgment. At least one file was selected for each CSA Program Alignment Architecture (PAA) sub-sub-program (SSP) that receives funding under the CSA Class G&C Program. An additional file was also included for the two SSPs with the largest budgets. For each selection, the amount of the agreement was taken into consideration.

In order to determine whether required procedures were applied, we looked at each file to determine if:

  • recipient eligibility criteria were applied;
  • cost eligibility criteria were applied;
  • initiative eligibility criteria were applied;
  • stacking provisions requirement were applied;
  • the method for determining the amount of the grant or contribution was respected;
  • application requirements were applied;
  • initiative evaluation criteria were applied;
  • official language requirements were applied;
  • criteria for establishing the duration of agreements were applied;
  • requirements regarding the production of reports on the results obtained were applied; and
  • ongoing eligibility requirements were applied.

The following are the detailed findings of the file review, for each aspect:

1. Eligibility requirements

After reviewing the 10 agreements, we found that, in general, eligibility criteria with respect to recipients, costs and initiatives were applied in compliance with the terms and conditions of the Program. However, we noted the following deficiencies:

  • One grant file contained no working documents, comments or opinions that could be used to determine the conclusions of the analysis of cost eligibility criteria. However, handwritten notes on the recipient's proposal suggest that an analysis had indeed been carried out.
  • In the case of one contribution file, no working documents for the purpose of applying recipient, cost and initiative eligibility criteria were initially included in the file. However, a signed copy of these items was obtained at the auditor's request.
  • The agreement for one grant file stipulated that a statement of continued eligibility was required for all subsequent payments after the initial payment. However, the second payment was made without this statement having been obtained. The wording of the clause of the agreement relating to this requirement could lead to confusion. The CSA G&C Centre of Expertise has informed us that this clause will be reviewed and clarified.

2. Determining the amount and duration of agreements

After reviewing the 10 agreements, we found that, for each agreement signed, the criteria for determining the amount and duration of the agreement had been applied in compliance with the terms and conditions of the Program. However, we noted the following deficiency:

  • In one file, the agreement stipulated that the recipient was required to submit confirmation of all sources of funding obtained for the project. This confirmation was not initially included in the file; however, it was subsequently obtained at the auditor's request.

3. Initiative evaluation

After reviewing the 10 agreements, we found in 9 cases out of 10 that the initiative evaluation criteria had been applied in accordance with the terms and conditions of the Program. However, we noted the following deficiency:

  • In one file, the evaluation documentation found in the file was that of the original proposal. No evaluation documentation pertaining to the amended proposal could be found in the files, despite the fact that the total amount of the proposal had increased from $150,000 to $400,000. Subsequent discussions with the auditee revealed that the amended proposal had indeed been evaluated, but that the evaluation had not been documented.

4. Requirements regarding funding applications

After reviewing the 10 agreements, we noted in 9 cases out of 10 that the funding application requirements had been applied in accordance with the terms and conditions of the Program, for each agreement signed. However, we noted the following deficiency:

  • In one case, the funding application submitted by the recipient was not included in the file. At the auditor's request, the missing proposal and the associated expenditure budget were obtained from the auditee; however, the application form was not obtained.

5. Requirements regarding the production of reports on results obtained

After reviewing the 10 agreements, we noted in 8 of the 10 files that the requirements for the production of reports on the results obtained had been applied in accordance with the terms and conditions of the Program, for each agreement signed. However, we noted the following deficiencies:

  • In one case, the final report for the period ending on March 31, 2012, was due by May 31, 2012; however, the report was not in the file at the time the file was given to the auditor. The recipient submitted the report at the auditor's request.
  • In another case, one of the progress reports was not in the file when the file was given to the auditor. The report concerned was obtained from the auditee at the auditor's request. It had not been signed by the recipient.

6. Official language requirements

After reviewing the 10 agreements, we noted that official language requirements had been applied in accordance with the terms and conditions of the Program, for each agreement signed.

Cause

Compliance with Program terms and conditions

There are a number of possible causes for the identified deficiencies. The main one is most likely a lack of knowledge, experience and/or training on the part of the employees responsible for applying the established controls. The lack of knowledge can be partially explained by the fact that, since the Program's expansion in 2009, there are more managers responsible for delivering the Program than there were before. Some of the identified deficiencies may also result from a lack of meticulousness in assembling the documentation for files.

Effect

Compliance with Program terms and conditions

Incomplete files that do not adequately support the decision making process.

Recommendation
  1. Apply existing controls systematically to ensure that files contain documentation in accordance with prescribed standards and procedures.
Identified Responsibility
Organization
  • Space Science and Technology Branch
  • Space Exploration Branch
  • Space Utilization Branch
Function

Directors General

Management's Response

The three branches are in agreement with this recommendation.

Management Action Plan
Details of the action plan

The Centre of Expertise will continue to provide leadership with respect to the implementation of controls in order to provide managers with tools to ensure that the CSA's files consistently contain documentation in accordance with prescribed standards and procedures.

Deadline: Ongoing

Mandatory training sessions will be developed and given to key staff members, identified by the sectors, who are involved in the management of G&C. These sessions will focus on the use of tools and the compiling of documentation for files.

Deadline: Developed by March 31, 2014 / Submitted by June 30, 2014

A review of the Centre of Expertise's roles and responsibilities will be undertaken in order to determine the best way of applying the existing controls.

Deadline: March 31, 2014

2.3.3 Compliance with TB policies and directives on financial management and transfer payments

Audit objective: A management framework is in place to ensure that the Class Grant and Contribution Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and that there is proper accountability reporting of the Program.

Findings
Criterion 3

The Program is carried out in accordance with TB policies and directives on financial management and transfer payments.

Condition

Further to our review, we noted that, in the majority of cases, the Program was carried out in accordance with TB policies and directives on financial management and transfer payments. However, we observed some deficiencies that require management's attention and for which we have made recommendations.

To determine whether the Program is implemented in accordance with TB policies and directives on financial management and transfer payments, we used the same files as those selected for the preceding criterion, and looked at the following aspects:

  • Whether for each payment, there was supporting documentation and certification provided by the manager with delegated financial authority (section 34 of the Financial Administration Act [FAA]);
  • Whether transfer payments to recipients were made in a timely manner, in accordance with the agreements;
  • Whether a recipient selection strategy had been developed for audit purposes, and whether a recipient audit plan had been drawn up, as required, and implemented, if applicable (solely in the case of contribution agreements);
  • Whether the funds had been previously committed, pursuant to section 32 of the FAA;
  • Whether payment verifications had been made, pursuant to section 33 of the FAA.

In our review of the files, we found the following deficiencies requiring management's attention:

  • In one contribution file, a number of documents required under the agreement for the purpose of issuing payments were missing. At the auditor's request, the auditee submitted all of the missing documents except one. In addition, the checklist that officers must provide as documentation prior to each payment was missing in the case of two of the six payments made;
  • In the case of one contribution agreement, at the time of the audit, no recipient risk analysis could be found in the file. At the auditor's request, the auditee submitted the risk analysis;
  • In the case of two grant agreements, the project activities were not defined in the funding agreements (except in the title);
  • In one grant agreement, the clause stipulating that the recipient's name, the amount of funding and the general nature of the activities being funded could be made public by the Government of Canada was missing;
  • Concerning the contribution agreements, the internal directive outlining the strategy for determining which recipients should be audited has yet to be drawn up and implemented.
Cause

Compliance with TB policies and directives

There are a number of possible causes for the identified deficiencies. The main cause is most likely a lack of knowledge, experience and/or training on the part of the employees responsible for applying the established controls. The lack of knowledge can be partially explained by the fact that, since the Program's expansion in 2009, there are more managers responsible for delivering the Program than there were before. Some of the identified deficiencies may also result from a lack of meticulousness in assembling the documentation for files.

Audit of recipients

The implementation of a recipient audit plan poses a number of challenges for the organization in terms of expertise and cost. Hiring an external firm to conduct a financial audit of a recipient, as suggested in the Directive on Transfer Payments, could represent a significant cost for the organization.

Effect

Incomplete files that do not adequately support the decision making process.

Failure to comply with the requirements of the Directive on Transfer Payments.

Recommendation
  1. Recommendation No. 1 also applies here: Apply existing controls systematically to ensure that files are documented in accordance with prescribed standards and procedures.
  2. Apply existing controls systematically to ensure that all relevant clauses and required project information are included in funding agreements, in accordance with the Directive on Transfer Payments.
  3. The internal directive concerning the auditing of recipients should be completed and an audit plan should be drafted and implemented.
Identified responsibility
Organization
  • Space Science and Technology Branch
  • Space Exploration Branch
  • Space Utilization Branch
Function

Directors General

Management's Response

The three branches are in agreement with these recommendations.

Management Action Plan
Details of the action plan
  1. The action plan in response to Recommendation No. 1 also applies here.

    Deadline: March 31 and June 30, 2014

  2. The Centre of Expertise will continue to ensure that all relevant clauses are included in funding agreement templates, and that all required project information is entered in the agreements, in accordance with the Directive on Transfer Payments, prior to the signing of the agreement by the Director General.

    In fact, these controls have already been in place since February 2012, further to the request submitted to the Centre of Expertise by the directors general, and following the implementation of the Policy on Internal Control. The Centre of Expertise is currently reviewing the content of each draft agreement (over $25K) and is working with the CSA's Finance Directorate to ensure compliance with TB policies and directives on financial management and transfer payments, prior to the signing of the agreements by the Director General of the sector concerned.

    Deadline: Already completed

  3. The Centre of Expertise will draft the final version of the internal directive regarding recipient audits and have it approved by the G&C Committee.

    Deadline: March 31, 2014

    If necessary, the Centre of Expertise will draw up a recipient audit plan, to be implemented as required.

    In addition, the recipient risk management framework and associated mitigation measures may be adjusted accordingly.

    Deadline: June 30, 2014

2.3.4 Accountability and performance measurement

Audit objective: A management framework is in place to ensure that the Class Grant and Contribution Program is managed in accordance with relevant legislation and policies and the approved terms and conditions of the Program, and that there is proper accountability reporting of the Program.

Findings
Criterion 4

There is accountability reporting of the auditee's operations, and results are measured.

Condition

Accountability

We found that there was accountability reporting of all program activities. The accountability reported is done in two ways: individual accountability reporting of each PAA SSP that receives G&Cs, and overall accountability reporting of the G&C Program as the umbrella program.

Individual accountability reporting: At the beginning of the fiscal year, SSP managers identify expected results in their work plans, as part of the CSA integrated management cycle. Each SSP manager submits detailed operational plans describing the activities to be carried out during the coming year, along with the resources that will be allocated to these activities, the desired outputs and the performance targets. Follow-up is done at mid-year and at year-end. The results will later be reported in the Departmental Performance Reports (DPRs) of their respective branches.

Overall accountability reporting: The results of the G&C Program are compiled by the CSA's G&C Centre of Expertise and presented in the DPR in a separate appendix. The TB requires this separate presentation for all transfer payment programs involving more than $5 million. A breakdown of the results of the Class Program is also to be included in the work plans for the current fiscal year (2013–2014).

Performance measurement strategy

In accordance with the Directive on the Evaluation Function, a performance measurement strategy (PMS) has been in place since 2009. This strategy allows for the systematic monitoring and measurement of the Program's results, as well as for the production of useful and relevant reports for decision making and the gathering of credible and reliable information to effectively support the Program's evaluation. As stipulated in section 6.1.4 of the Directive on the Evaluation Function, which has been in force since April 1, 2009, "developing, implementing and monitoring ongoing performance measurement strategies for programs is the responsibility of program managers." However, the strategy was not fully implemented, as some indicators posed a challenge in terms of the data gathering methodology and were not adopted as planned in 2010. We noted that approximately 80% of the indicators were measured as originally planned. Over the past year, a lot of work has been done into updating the strategy and reviewing the problematic indicators. A new version of the strategy, including the new indicators, is currently in consultation. A new data gathering system was also set up in recent months to facilitate communication with recipients and improve data management. Performance measurement tools are now combined with project monitoring tools.

Overall, even though the performance measurement strategy is in the process of being improved, we find that the strategy is being implemented and that it allows for the measurement of a significant portion of the Program's expected results. The changes currently being made should help improve the strategy's effectiveness.

Cause

N/A

Effect

N/A

Recommendation

N/A

Identified responsibility
Organization

N/A

Function

N/A

Management's Response

N/A

Management Action Plan
Details of the action plan

N/A

Deadline: N/A