Language selection

Search

Top of page

Audit of the Management Framework for the Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology

Audit Report

Project No. 18/19 01–04

Prepared by
Audit and Evaluation Directorate

Table of contents

Acronyms

AO

Announcement of Opportunity

ARLU/MJANR

Annual Reference Level Update

CSA

Canadian Space Agency

EC

Executive Committee

EOAU/AUOT

Earth Observation Applications and Utilizations

EPMO

Enterprise Project Management Office

ESA

European Space Agency

FAA

Financial Administration Act

FAST LEAD/VITES

Flights and Fieldwork for the Advancement of Science and Technology — Lunar Exploration Analogue Deployment

G&C

Grants and Contributions

G&C COE

Grant & Contribution Centre of Expertise

IAC

International Astronautical Congress

IGMF/CGSI

Investment Governance and Monitoring Framework

IIRB

Integrated Investment Review Board

IT

Information Technology

PIP

Programs and Integrated Planning

PSRS

Project Status Report System

RBAP

Risk-Based Audit Plan

SE

Space Exploration

SST

Space Science and Technology

STDP/PDTS

Space Technology Development Program

SU

Space Utilization

TB

Treasury Board

TBS

Treasury Board Secretariat

1.0 Summary

1.1 Audit objective

The objective of the audit project was to determine whether a management framework is in place to enable the Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology (the Program) to achieve its objectives and comply with the applicable laws, policies, regulations and guidelines issued by the Canadian Space Agency (CSA) and central agencies.

1.2 Audit opinion

In our opinion, the framework in place enables the Program to achieve its objectives and comply with the applicable laws, policies, regulations and guidelines issued by the CSA and central agencies. However, opportunities for improvement were identified with respect to certain processes.

1.3 Statement of assurance

As Chief Audit Executive, I am of the opinion that sufficient and appropriate audit procedures have been followed and evidence gathered to support the accuracy of the opinion provided in this report. This opinion is based on a comparison of the conditions, as they existed at the time of the audit, against pre-established audit criteria determined in consultation with the management.This opinion is only applicable to the particular entity being audited. The evidence was gathered in compliance with Treasury Board (TB) policy, directives and standards for internal audit. The procedures followed are in accordance with the professional standards of the Institute of Internal Auditors. The evidence gathered is sufficient to convince senior management of the validity of the opinion derived from the internal audit.

1.4 Summary of findings

The audit showed that the CSA Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology is managed in accordance with the Policy on Transfer Payments, the Financial Administration Act (FAA) and the program terms and conditions. We examined a sample of 10 grants and contributions (G&C) agreements and determined that they complied with the rules in effect. Furthermore, several consultations are conducted in the planning stage, and a decision approval process is in place. G&C agreements are issued via processes that are structured and well documented. Once they have been issued, agreements are monitored adequately and short-term results are followed.

However, we did identify opportunities for improvement in the management of service standards regarding response time to render a decision following an Announcement of Opportunity (AO), the establishment of a methodology for conducting recipient audits, and the obtention of information on results following G&C agreements. Our findings led to the formulation of the following opportunities for improvement:

  1. Develop and implement a methodology for recipient audit
  2. Review the process for establishing service standards, and, as necessary, the service standards regarding response time to render a decision following an AO

Original signed by Dominique Breden

Signature of the Chief Audit Executive

Audit team members

  • Dany Fortin
  • Louis Martel
  • Fatima Raveen
  • Johanna Gailer

2.0 Audit report

2.1 Background

The objective of the Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology is, firstly, to support knowledge growth and innovation in the Agency's priority areas (research component), and secondly, to increase the awareness and participation of Canadians in space-related disciplines and activities (awareness and learning component).

The research component aims to support the development of science and technology, foster the continuing development of a critical mass of researchers and highly qualified people in Canada, and support information-gathering, studies and research related to space in light of the Agency's priorities. The awareness and learning component aims to provide learning opportunities to Canadian students in various space-related disciplines, support the operations of organizations dedicated to space research and education, and increase awareness of Canadian space science and technology among Canadian students and their participation in related activities.

When the audit was conducted, approximately 100 contribution agreements and 159 grant agreements were active at the CSA. AOs and the issuance and management of agreements under the program are handled by the various sectors of the CSA. The Grant & Contribution Centre of Expertise (G&C COE), under the direction of the Finance Directorate, provides advice and support to the sectors. Among other things, the G&C COE is responsible for developing and standardizing processes, tools and infrastructures for managing G&C. The G&C COE is also responsible for certain G&C monitoring, reviewing and reporting activities.

G&C Expenditures Per Year
Expenditures -
ActualTable note 1
$M
-
ActualTable note 1
$M
-
ActualTable note 1
$M
-
ActualTable note 1
$M
-
ActualTable note 1
$M
-
ActualTable note 1
$M
-
ForecastTable note 2
$M
-
ForecastTable note 2
$M
-
ForecastTable note 2
$M
Grants 6.3 7.0 6.3 9.1 8.7 10.4 10.0 8.5 9.4
Contributions 1.5 4.1 10.5 11.9 10.5 16.0 16.6 16.8 16.8
Total 7.8 11.1 16.8 21.0 19.2 26.4 26.6 25.3 26.2

Sources:

2.2 Audit objective, scope and approach

Objective

This audit project is part of the - to - Risk-Based Audit Plan (RBAP) approved by the agency's Audit Committee. The objective of this audit was to determine whether a management framework was in place to enable the program to achieve its objectives and comply with the applicable laws, policies, regulations and guidelines issued by the CSA and central agencies.

Scope

The audit project focused on all program expenditures, as well as the systems and procedures for activity planning, control and accountability. The audit examined activities carried outfrom to .

Approach

The audit criteria were established in accordance with best management practices as well as requirements set out in CSA and central agency guidelines, regulations, policies and legislation. The audit involved various processes including interviews, reviews of documents, operating expenses and contract files.

For 6 contributions and 4 grants, we reviewed compliance with:

  • the Treasury Board Secretariat (TBS) Directive on Transfer Payments
  • the agency's program's terms and conditions
  • the Financial Administration Act (sect. 32, 34 and 33)

The samples were selected at the discretion of the auditors in accordance with the following criteria:

  • type of funding (prorated based on monetary value between G&C)
  • sectors (prorated based on monetary value between sectors)
  • type of recipient (prorated between companies and academic institutions)
  • monetary value per transaction

For one AO, we examined the cycle made up of:

  • the preparation and publication of the AO
  • the preselection, evaluation and selection of proposals
  • the issuance and following of grant agreements

It should be noted that the audit objective and audit criteria were discussed with management. The audit criteria and sub-criteria are set out in Appendix A.

2.3 Findings, recommendations and management reactions

Expected results:

To determine whether the management framework in place enables the program to achieve its objectives and comply with the applicable laws, policies, regulations and guidelines issued by the CSA and central agencies, we expected to find the following elements:

  • effectively planned activities;
  • monitored activities and resources used;
  • accountability for operations.

2.3.1 Operation planning

Audit objective: The objective of the audit was to determine whether a management framework was in place to enable the Program to achieve its objectives and comply with the applicable laws, policies, regulations and guidelines issued by the CSA and central agencies.

Findings
Criterion 1

Program activities are planned appropriately.

Condition

Conclusion on the criterion:

The audit showed that program operations are planned appropriately. Stakeholder needs are identified regularly, and the investment areas selected take the priorities identified into account. Investment plans are prepared, approved and reviewed regularly.

Investment planning

Identification of needs

The programmatic sectors regularly consult the industry, academic institutions and national and international partners to identify needs. Consultations are conducted through advisory committees, workshops and working groups. The programmatic sectors also participate in national and international conferences, and have regular contact with international partners. Environmental analyses are conducted to identify context, situate Canada in terms of trends in the space domain and establish the profile of Canadian resources. There are also several internal consultations among the sectors of the CSA.

Prioritization and approval

The objective of the program is to support knowledge growth and innovation and increase the awareness and participation of Canadians in space-related activities, prioritization criteria vary with the scientific discipline. Preparation work for investment plans differs from one programmatic sector to another depending on the activity. Some sectors coordinate their operations in collaboration with other space agencies, and roadmaps are prepared on the basis of international opportunities and planned missions.

Each sector presents its director general with the investments it considers to have the greatest advantages for the Canadian space industry, the Canadian academic environment, Canada as a whole and the CSA. The investments approved by the director general are then presented to the Integrated Investment Review Board (IIRB) for approval, followed by AO preparation. Unsolicited proposals are all presented to the IIRB for approval.

The program budget was allocated among the sectors a few years ago, and it is renewed from year to year with some variations. Optimal planning of available resources should include a periodic review in accordance with any changes to organizational priorities and expected results.

Financial planning

A review of the budgets and expenditure forecasts for the ongoing fiscal year is conducted monthly using the SAP financial system. Potential surpluses are identified during the year and redistributed to another G&C program at the CSA.

For -, the workplan total went from $21.6 M at the beginning of the fiscal year to $19.2 M at the end, with a decrease of $2.4 M (11.0%). For -, the workplan total went from $23.8 M at the beginning of the fiscal year to $26.4 M at the end, with an increase of $2.6 M (11%). The increase occurred due to additional funds being obtained for new initiatives approved over the course of the year. Without those additional funds, the potential surpluses identified over the course of the year and transferred to another program were approximately $2.5 M (10.5%), which is a similar proportion to the previous year.

Recommendation

N/A

Responsibility identified
Organization

N/A

Function

N/A

Management response

N/A

Management action plan
Action plan details

N/A

Deadline

N/A

2.3.2 Program operation control

Audit objective: The objective of the audit was to determine whether a management framework was in place to enable the Program to achieve its objectives and comply with the applicable laws, policies, regulations and guidelines issued by the CSA and central agencies.

Findings
Criterion 2

Program activities are monitored appropriately.

Condition

Conclusion on the criterion:

Our audit enabled us to conclude that program activities are monitored appropriately. Expenditures are authorized and made in accordance with applicable laws, regulations and program terms and conditions, and the transactions are accounted for appropriately. The preparation and publication of AOs, as well as the reception, analysis, evaluation and selection of proposals, follow structured and documented processes and procedures. Recipients are monitored on an ongoing basis.

However, areas for improvement were identified with respect to updating the documentation for procedures associated with G&C operations in accordance with the new CSA Investment Governance and Monitoring Framework (IGMF/CGSI) and establishing a methodology for auditing recipients.

Applicable laws, policies, regulations, guidelines and Program terms and conditions

Our audit enabled us to conclude that expenditures are approved and made in accordance with the Policy on Transfer Payments, the FAA and the program terms and conditions.

To reach this conclusion, we reviewed a sample of 10 agreements: 6 contribution agreements and 4 grant agreements. Our review showed that expenditure authority is exercised in accordance with delegated authority (FAA sect. 32), that expenditures are made in accordance with G&C agreements, that payments are supported by certifications to the effect that the terms of the agreements have been met, and that invoices are certified under the FAA sect. 34 by an authorized party. Furthermore, payments are certified under FAA sect. 33 by an authorized party, and expenditures are accounted for appropriately.

With respect to payments, we selected the last invoice submitted for each of the 10 agreements in our sample. We concluded that the amounts invoiced generally corresponded to the basis of payment in the agreement. However, in two cases, we noted that part of the payment had not been retained as indicated in the agreement (5% and 10%).

Announcement of opportunity and proposals

We found that the preparation and publication of AOs, as well as the reception, analysis, evaluation and selection of proposals, follow structured and documented processes and procedures. These processes and procedures allow for compliance with the program terms and conditions.

We examined all the activities conducted as part of an AO and concluded that the processes and procedures had been applied as planned. The prescribed checklists and templates were used. The preparation and posting work was supervised by the G&C COE, and the AO was approved by the IIRB. Proposals were selected in accordance with the criteria set out in the AO, the results were communicated to the applicants and the agreements were signed by the appropriate delegation level. The G&C COE provided support to the team responsible for the AO throughout the process.

Recipient monitoring

In the course of our audit, we found that agreements were monitored on an ongoing basis through progress reports, annual reports and final reports submitted by recipients.

The annual reports and final reports are documented in the CSA G&C management infrastructure (UNITAS). The information is entered by recipients via the web portal using specific templates, and then transferred electronically into the UNITAS system. The annual progress report compares the results to the project's general objectives, so the project's ongoing eligibility is periodically evaluated. The monitoring is conducted by the project authority with the assistance of the technical authority.

Other progress reports are submitted during the year for the purpose of monitoring agreements. These reports are submitted by recipients in hard copy and/or digital format, but they are not transferred into the UNITAS system. Entering these reports into the UNITAS system would allow for complete information to be obtained from a single source.

Recipient audits

The Directive on Transfer Payments requires that a time be identified to conduct recipient audits, and that an audit plan be developed and implemented based on risk analysis. To meet that requirement, the CSA has produced two documents to direct risk assessment. Currently, no recipient audits have been conducted at the CSA, and none are planned. Furthermore, no plans have been developed and no audit methodology is in place to support such a measure. A proper plan developed using a precise methodology would enable the CSA to meet the requirements of the directive.

Roles and responsibilities

The roles and responsibilities for Program governance have been defined. The main stakeholders are the various branches of the CSA, the G&C COE and the G&C steering committee formed in .

Among other things, the sectors are responsible for planning, implementing and evaluating AO processes, and for issuing and managing program agreements.

The G&C COE is responsible for providing advice and support to the sectors throughout the AO and agreement issuance processes. The G&C COE is also responsible for developing, standardizing, implementing and maintaining G&C processes, management tools and infrastructures, and for monitoring and reporting on certain aspects of G&C. The G&C COE provides support to the G&C steering committee and to the IIRB in its approval decisions for G&C projects.

Chaired by the CSA Vice-President, the G&C steering committee has two main roles. it supervises G&C governance and management for the CSA, and it provides strategic advice on the use of funding.

Processes and procedures

Processes, procedures, guides and templates have been developed to support G&C activities, and are available through the G&C COE Livelink portal. However, we noted that some have not been updated since the changes were made to the CSA governance processes. Updates to the operational guidelines, processes and procedures would enable them to reflect current practices.

G&C management infrastructure (UNITAS)

The UNITAS system is a digital G&C management infrastructure. It was developed and is supported by the information technology (IT) sector of the CSA. UNITAS has been implemented gradually since -. Over the years, since the system has been in use, there have been new requests for added G&C functionalities. The various functionalities of the UNITAS system have been grouped into modules. An initial development plan for the UNITAS system was established in , and updates have been conducted over the years. Some elements of the plan have been fully implemented, and others only partially or not at all.

The development of the UNITAS system is ongoing. The work being done involves electronic functions related to the preparation and posting of AOs, reception of proposals, management of service standards and proactive disclosure of agreements. Investments were approximately $200,000 per year in - and -. These amounts are primarily made up of time spent by IT and professional services employees. Every year, UNITAS development needs are sent to the IT sector by the G&C COE and are analyzed. Due to limited resources, development priorities are established together with the G&C COE. Future developments are planned or discussed, including the automation of agreements and amendments, the automation of the documentation required for governance, a control point module, the upgrading of the agreement management dashboard, and availability of historical data on funded organizations.

With respect to the governance of the UNITAS system developments, several control measures are applied, and the work is guided. Annual implementation plans are established with deadlines. An annual plan is made for the distribution of IT labour based on the activities set out, and needs for professional services are established. Futhermore, documents showing system development and progress are produced regularly. Meetings are held between IT and the G&C COE to ensure that the development work meets expectations. The G&C steering committee has been consulted annually for the last two years, and consultations to determine the applicability of the IGMF/CGSI are ongoing.

Recommendation
  1. Develop and implement a methodology for recipient audits
Responsibility identified
Organization

Finance Directorate - G&C COE

Function

Manager, Grant & Contribution Centre of Expertise

Management response

We agree with the recommendation.

Management action plan
Action plan details

Recommendation #1:

  1. The G&C COE will develop a recipient audit plan
  2. The G&C COE will implement the steps required to carry out the plan (e.g. select files to review, audit recipients, draft a final report)
Deadline

2.3.3 Accountability

Audit objective: The objective of the audit was to determine whether a management framework was in place to enable the Program to achieve its objectives and comply with the applicable laws, policies, regulations and guidelines issued by the CSA and central agencies.

Findings
Criterion 3

Program operations are subject to accountability.

Condition

Conclusion on the criterion:

The audit showed that the CSA is generally in compliance with the reporting obligations set out in the applicable laws, policies, regulations and guidelines issued by the CSA and central agencies.

However, we did identify some deficiencies in terms of proactive disclosure and establishment of service standards. We also identified an opportunity for improvement with respect to obtaining information on the results of the program following the issuance of G&C.

Reporting obligations

The main external reporting obligations for the program are as follows: proactive disclosure of G&C issuance; establishment and communication of service standards; and disclosure of results in terms of public accounts, plans and departmental results. We found that the external reporting obligations are generally met. However, we did identify the deficiencies below.

Proactive disclosure

For 1 agreement of the 13 reviewed, its proactive disclosure was not included on the CSA website. According to the sector responsible, there was an oversight in entering the information into the CSA's proactive disclosure system. Following the audit, this agreement was published with the proactive disclosures for the first quarter of / with a "late publication" note. The sector responsible also states that it intends to develop a procedure to validate proactive disclosures once they have been published.

According to the G&C COE, improvements are in development with respect to the proactive disclosure process. A new functionality in the UNITAS G&C infrastructure should allow for the process to be automated from the AO. The new process should eliminate the risk of oversights by the various sectors that, until now, have been responsible for entering the information manually into a separate system. As the appropriate actions are being taken, no recommendations have been made in this report.

Service standards

With respect to G&C management framework, the Policy on Transfer Payments requires that reasonable and practical service standards be established for services provided to Canadians. The results are reported under the Management Accountability Framework (MAF) and published on the CSA website. To meet that requirement, the CSA has established service standards for the reception of proposals, decision timelines following the reception of proposals, and payment processing timelines following claims. Service standards are announced publically when the AO is issued.

The service standard results for -, - and - were examined. One of the standards was not met: the response time for rendering a decision on proposals received following an AO. The standard established by the CSA, which can vary between AOs, is 26 to 46 weeks. The target is that 80% of the proposals received meet that standard. For the years reviewed, as shown in the table below, the target was met only once, in -. The reasons given are an unrealistic assessment of the timeline and the deadline for the approval of proposals selected.

Service standards Performance
Target - - - -
Acknowledgement of applications 80% 96% 100% 88% 86%
Proposal selection decisions 80% 58% 39% 84% 16%
Issuance of payments 80% 97% 88% 88% N/A

Results of G&C initiatives

Short-term results

G&C agreements include the disclosure of results obtained following the implementation of the activities associated with the agreements. Recipients must demonstrate the work completed after funding has been obtained. Reports are required, results and data on indicators must be communicated to the CSA. The CSA compiles the results, allowing it to meet its aforementioned reporting obligations.

Long-term results

In the audit of the Space Technology Development Program (STDP/PDTS) in , an opportunity for improvement was identified with respect to obtaining information on results following the issuance of G&C. It involved identifying the more long-term impacts and benefits that the agreements had generated for recipients. We considered that more detailed statistics and reports could better support planning and decision-making for future investments. An action plan is being implemented; it involves a mechanism for collecting information using surveys and communicating results in report form. We believe that a similar approach could be implemented to evaluate the results of the program's initiatives.

Because a recommendation was already made as part of the STDP/PDTS audit, we suggest that the G&C COE follow the progress of the action plan implementation and determine how it could be beneficial to the program.

Recommendation
  1. Review the service standard establishment process and, as necessary, the service standards for the response time to render a decision following an AO.
Responsibility identified
Organization
  • Finance Directorate – G&C COE
  • Programmatic sectors or G&C users
Function
  • Manager, Grant & Contribution Centre of Expertise
  • Directors and managers, Programmatic sectors or G&C users
Management response

We agree with the recommendation.

Management action plan
Action plan details
  1. The G&C COE will add a specific component regarding service standards to the IIRB submission template to facilitate decision-making for senior management.
  2. The G&C COE will review the process for establishing service standards with the sectors.
  3. The G&C COE will update the CSA guide on service standards.
Deadline

Appendix A – Terms of reference

Audit objective: The objective of the audit was to determine whether a management framework was in place to enable the Program to achieve its objectives and comply with the applicable laws, policies, regulations and guidelines issued by the CSA and central agencies.

Audit criteria Audit sub-criteria
  • Sub-criterion met 
  • Sub-criterion partially met 
  • Sub-criterion not met 
Criterion 1: Program operations are planned appropriately. Sub-criterion 1.1: Investment priorities are established following a complete analysis of opportunities. Sub-criterion met
Sub-criterion 1.2: Financial resources are planned in accordance with the activities set out. Sub-criterion met
Criterion 2: Program operations are controlled appropriately. Sub-criterion 2.1: Roles and responsibilities are defined. Sub-criterion met
Sub-criterion 2.2: Publishing of AOs, selection of proposals and issuance of agreements are in compliance with Program terms and conditions and applicable laws, policies, regulations and guidelines. Sub-criterion met
Sub-criterion 2.3: Agreement management is in compliance with Program terms and conditions and applicable laws, policies, regulations and guidelines. Sub-criterion met
Sub-criterion 2.4: Results and activities of agreement recipients are monitored. Sub-criterion partially met
Criterion 3: Program operations are subject to accountability. Sub-criterion 3.1: Reporting obligations to central agencies are met. Sub-criterion partially met
Sub-criterion 3.2: Initiative results are reported. Sub-criterion met
Date modified: