Language selection

Search

Top of page

Audit of the Management Framework of the Canadian Space Agency's (CSA) Space Astronomy and Planetary Missions Programs

Audit Report

Project No. 17/18 01–01

Prepared by
Audit and Evaluation Directorate

Table of contents

1.0 Summary

1.1 Audit objective

The objective of this audit is to determine whether a management framework is in place to enable the sub-sub-programs' objectives to be achieved and to ensure compliance with relevant CSA and central agencies acts, policies, regulations and guidelines.

1.2 Audit opinion

In our opinion, the management framework in place allows the sub-sub-programs' to achieve its objectives and to comply with relevant acts, policies, regulations and guidelines issued by the CSA and central agencies.

1.3 Statement of assurance

As Chief Audit Executive, I am of the opinion that sufficient and appropriate audit procedures were followed and that audit evidence was collected to support the accuracy of the opinion provided in this report. This opinion is based on a comparison of the conditions, as they existed at the time of the audit, against the pre-established audit criteria determined in consultation with management. The opinion is applicable only to the entity examined. The evidence was gathered in compliance with Treasury Board's (TB) internal audit policy, directives and standards. The procedures followed are in accordance with the professional standards of the Institute of Internal Auditors. The evidence gathered is sufficient to convince senior management of the validity of the opinion derived from the internal audit.

1.4 Summary of findings

Our audit revealed that Space Astronomy Missions (1.2.2.1) and Planetary Missions (1.2.2.2) sub-sub-programs have established best practices to ensure that CSA operations are planned effectively and that the resources used are adequately monitored. Among other things, we noted that investment priorities were determined based on an opportunity analysis as well as partners' needs, and that activities and results are monitored throughout projects' life cycle. In addition, we found that an appropriate accountability process is carried out periodically on all activities.

We also found that the sub-sub-programs generally complied with relevant CSA and central agencies' acts, policies, regulations and guidelines. We reviewed samples of contract awards, operating expenses, capital spending, travel, and grant and contribution agreements, and these were all in compliance with applicable rules.

Original signed by Dominique Breden

Chief Audit Executive Signature

Members of audit team:

  • Dany Fortin
  • Louis Martel
  • Fatima Raveen

2.0 Audit report

2.1 Background

The sub-sub-programs (SSPs) covered by this audit are Space Astronomy Missions (1.2.2.1) and Planetary Missions (1.2.2.2). SSP 1.2.2.1 encompasses the definition, design, technology development, implementation and use of Canadian scientific instruments and signature technologies made available to Canadian and international space astronomy missions. These activities are necessary to perform space astronomy investigations and generate data and new knowledge about the universe. SSP 1.2.2.2 encompasses the definition, design, technology development, implementation and use of Canadian scientific instruments and signature technologies made available to international exploration missions. These activities are necessary to reach exploration destinations such as planets and asteroids or new exploration platforms to conduct planetary science investigations, to generate data and new knowledge and to conduct engineering and/or planetary resource management activities. These SSP activities are carried out in collaboration with foreign space agencies, the international space exploration community and Government of Canada (GoC) organizations, and through consultations with the Canadian astronomical community. This collaborative effort takes shape under contracts and/or international partnership agreements. These SSPs are also funded through the Class Grant and Contribution (G&C) Program to Support Research, Awareness and Learning.

The following tables present financial data and the number of employees for the SSPs.

-12
$M
-13
$M
-14
$M
-15
$M
-16
$M
-17
$M
Space Astronomy (1.2.2.1) 14.0 15.0 11.7 14.0 7.3 3.4
Planetary Missions (1.2.2.2) 4.7 8.5 18.9 10.4 7.4 4.0
18.7 23.5 30.6 24.4 14.7 7.4
-12
FTEs
-13
FTEs
-14
FTEs
-15
FTEs
-16
FTEs
-17
FTEs
Space Astronomy (1.2.2.1) 17.7 16.7 12.0 6.0 6.0 3.7
Planetary Missions (1.2.2.2) 11.2 9.9 10.6 9.5 9.6 6.4
28.9 26.6 22.6 15.5 15.6 10.1

2.2 Audit objective, scope and method

Objective

This audit project is part of the - to - Risk-Based Audit Plan (RBAP) that was approved by the CSA Audit Committee. Given that these two programs are grouped together for program evaluation purposes, the same has been done for the purposes of this audit. This audit is justified by the fact that these SSPs significantly contribute to the organization's objectives. Furthermore, these SSPs had never been audited before.

Scope

The audit focused on all SSP expenditures as well as planning, monitoring and accountability systems and processes. The audit covered activities carried out between , and .

Results measurement was not covered in this audit because the SSPs were undergoing program evaluation by the CSA's Evaluation Function at the same time.

Method

The audit criteria were established in accordance with best management practices as well as requirements set out in CSA and central agencies acts, policies, regulations and guidelines. The audit involved various audit processes including interviews as well as reviews of documents, operating expenses and contract files.

We reviewed:

  • The management of two projects with a high monetary value (JWST and OLA)
  • 3 grant and contribution awards
  • 7 payments related to operating costs or capital expenditures (about $1.35M)
  • 2 grant payments ($125K)
  • 1 contribution payment (about $30K)
  • 4 trips ($5.6K)

The samples were selected at the discretion of the auditors. The audit criteria and sub-criteria are shown in Appendix A.

It should be noted that the audit objective and criteria were discussed with management.

2.3 Findings, recommendations and management response

Expected outcomes

To determine whether the management framework is in place enables the SSP objectives to be achieved and to ensure compliance with relevant CSA and central agencies' policies, regulations and guidelines, we expected to find the following:

  • operations are planned effectively;
  • operations and use of resources are controlled; and
  • activities are subject to accountability.

2.3.1 Sub-sub-program activities planning

Audit objective: The objective of this audit was to determine whether a management framework is in place to enable the sub-sub-programs' objectives to be achieved and to ensure compliance with relevant CSA and central agencies acts, policies, regulation and guidelines.

Finding(s)
Criterion 1

The sub-sub-programs' activities are planned appropriately.

Condition

Conclusion about the criterion:

The audit showed that the sub-sub-programs' operations are planned appropriately.

Investment priorities are determined based on a complete analysis of opportunities.

We found that investment priorities for Space Astronomy and Planetary Missions are established once opportunities have been identified and analyzed. The resulting projects take account of affected scientific communities' needs and priorities, in addition to being in line with CSA and GoC priorities. The process followed is in accordance with the CSA's Investment Governance and Monitoring Framework (IGMF).

CSA's Investment Governance and Monitoring Framework (IGMF)

The two aspects of the IGMF that were taken into consideration for this sub-criterion were the "idea generation" and "pre-phase 0" project phases. The purpose of the "idea generation" phase is to identify a proposed initiative or idea that recognizes a need or opportunity. The information required at this stage consists of a high-level description of a technology development need or opportunity. During this stage, options are identified, concept studies are conducted, and investments can be planned. The work done during this phase determines whether and, if applicable, how the CSA can carry out a future project, either alone or with a partner. Viable options are identified. At the end of this phase, approval is required from the CSA's Integrated Investment Review Board (IIRB) before the next phase can begin. Given the exploratory nature of the work carried out during this phase, it does not necessarily result in a specific project.

Identification of potential missions and ideas

Ideas are identified to establish new missions that are based on an analysis of both national and international priorities for space exploration. The CSA derives ideas for future space astronomy and planetary missions from scientific communities involved in those fields, including but not limited to the Canadian Astronomical Society (CASCA), Joint Space Exploration Consultation Committee (JSECC), Planetary Exploration Consultation Committee (PECC), and Canadian Space Exploration Workshop (CSEW). Other ideas emerge from discussions with international partners at conferences and through advisory groups, as well as from direct dialogue with foreign space agencies, in addition to consulting their space strategies.

In , the CSA helped establish topical teams devoted to space science to identify scientific objectives as well as related activities and research in space exploration fields (space astronomy and planetary exploration). Topical team members mainly consisted of university and industry professionals. Each topical team had to submit a report summarizing results from consultations and discussions within the team, in accordance with the CSA's prioritization criteria. Each team was tasked with identifying opportunities, priorities and steps required for that field to conduct space missions over the next decade and beyond, based on work conducted by Canada's scientific community, among other things.

Selection of potential missions and ideas

Based on a prioritization approach, consultations are held on a regular basis with the Space Exploration sector's directors, technical experts, managers and scientists to identify and compile a list of priority ideas. Ideas related to space astronomy missions and planetary missions that, first of all, are consistent with Government of Canada and CSA priorities, and that are of great interest to Canada's space sector, priorities are identified. These results prepare Canada for providing informed contribution options to mission partners.

The CSA's space exploration planning group establishes a pool of potential missions based on ideas that have passed the initial screening. This list of potential missions is updated twice a year to include new ideas and remove those that have become obsolete. The planning group recommends ideas that it considers correspond with the CSA's priorities, available budgetary allocations and scheduling. The strategic planning group proposes investigations, consultations and studies to identify, develop and validate the options provided in support of strategic planning for Space Exploration Branch activities as well as short-term decision making. Everything is compiled in a document entitled "Space Exploration Preparatory Activity Priorities", which is presented to the Space Exploration Management Committee (SEMC) once a year. The Director General can approve the document only once approval has been received from all committee members. IIRB approval is then required before the actions proposed by the planning group can be carried out. Consequently, the Director General, Space Exploration, presents a document to the IIRB that is similar to the one presented to the SEMC.

To validate that investment priorities are established based on a complete opportunity analysis, we reviewed documentation from a mission (WFIRST) included on the CSA's list of potential missions. We found that all steps taken were consistent with the process described above, and that a comprehensive document had been prepared describing the analyses conducted, alignment with priorities and expected benefits.

The "Mission Datafile Prototype" register was updated with the 10 or so potential missions listed based on identified ideas. Key information for identified ideas was also included in the register in a set format.

This register is separated into three parts: the first part lists ideas that have been selected; the second lists ideas that may be selected following additional research; and the third lists rejected ideas.

The information recorded in the register related to identified ideas includes but is not limited to the source of the idea; the mission's name, description, objectives, partners; key dates; money the CSA has already invested; and potential ways the CSA can contribute. One section of the register also includes comments as well as occasional references to stakeholder priorities.

This register serves as a basis for compiling the pool of potential missions (mission roadmap).

This document could be more comprehensive by systematically including how missions align with CSA priorities as well as expected benefits. This would provide clearer justification for the decision on whether to participate in these missions.

Plans for financial resources have been made for planned activities.

We found that financial planning was done for planned activities in terms of both operating costs and capital expenditures. Plans were made taking into account both overall expenses and the specifics of each mission. Planning was continuously updated, and some capital expenditures were carried forward to next year based on project circumstances, in compliance with the established criteria. For -, we found that 27% of planned capital expenditures had been carried over to -. As for operating costs, a budget surplus of slightly over 20% was noted for -, based on a comparison of the revised budget with actual spending. Management provided justifications to explain this variance, and preventive measures have been implemented for - to ensure that planned spending is more consistent with actual spending.

Recommendation

No recommendation.

Identified Responsibility
Organization

N/A

Function

N/A

Management Response

N/A

Management Action Plan
Action plan details

N/A

Deadline: N/A

2.3.2 Sub-Sub-Program activities monitoring

Audit objective: The objective of this audit was to determine whether a management framework is in place to enable the sub-sub-programs' objectives to be achieved and to ensure compliance with relevant CSA and central agencies acts, policies, regulation and guidelines.

Finding(s)
Criterion 2

The sub-sub-programs' activities are monitored appropriately.

Condition

Conclusion about the criterion:

The audit demonstrated that the sub-sub-programs' activities are monitored appropriately.

Roles and responsibilities are clearly defined.

We found that roles and responsibilities are well defined and communicated. A reorganization of the Space Exploration Branch was carried out during the - fiscal year. A new directorate was created that encompasses all management planning activities. Notably, the position of Director, Space Exploration Strategic Planning was created. A new Director, Space Exploration Development was also appointed. The number of Space Astronomy and Planetary Missions SSP managers decreased from four to two. The job description for managers was also re-written to establish new roles and responsibilities.

Senior engineers are project managers who report to the Manager, Projects/Programs Portfolio, who in turn reports to the Director, Space Exploration Development.

Projects are monitored and tracked as per the CSA's Investment Governance and Monitoring Framework and Project Management Policy.

We found that projects were monitored and tracked as per the CSA's IGMF and Project Management Policy.

We selected two active projects (JWST and OLA)—one for each SSP—to observe the practices in place. We noted that both projects are managed based on the assessment of their level of complexity and risk. The expected results are clearly established, documented and demonstrated throughout the project's life cycle. Activities are monitored and tracked. Appropriate approvals are obtained when required. In addition, reliable information on project status is produced when appropriate to support decision making. The results of activities are compared with their expected results, and corrective action is determined and taken promptly, when necessary. Cost and scheduling forecasts are updated periodically.

Spending authorizations are controlled, expenditures are approved and made in accordance with laws and regulations, and transactions are recorded appropriately.

We reviewed a sample of 14 financial transactions related to operating costs, capital expenditures, and grants and contributions. The selection was based on the various projects and their monetary value. The total value of expenses reviewed was $1.5 million, representing approximately 20% of total expenses for the - fiscal year.

We found that the program complies with relevant CSA and central agencies acts, policies, regulations and guidelines. Operations are authorized and approved in accordance with the Financial Administration Act (sections 32, 33, and 34). Transactions are made on the basis of payment relating to contractual agreements, legislation and guidelines. In addition, the trips reviewed were justified, and the most economical means were used.

In one reviewed transaction, however, an additional $50K in funding was verbally authorized for a collaborative agreement. In our opinion, authorization should have been provided in writing. The associated risk was quite limited given that the agreement was with another GoC department. Measures were implemented to specify what process to follow in these cases and thereby ensure that collaborative agreements are amended, if necessary.

Grants and contributions are awarded as per the TBS Policy, Directive and Guideline on Transfer Payments as well as the terms and conditions of the CSA's grant and contribution programs.

For grant and contribution awards, we selected a sample of two grants and one contribution. Based on all documentation reviewed, the grants and contribution were awarded as per the TBS Policy, Directive and Guideline on Transfer Payments. They were also authorized by the IIRB.

Recommendation

No recommendation.

Identified Responsibility
Organization

N/A

Function

N/A

Management Response

N/A

Management Action Plan
Action plan details

N/A

Deadline: N/A

2.3.3 Accountability

Audit objective: The objective of this audit was to determine whether a management framework is in place to enable the sub-sub-programs' objectives to be achieved and to ensure compliance with relevant CSA and central agencies acts, policies, regulations and guidelines.

Finding(s)
Criterion 3

The sub-sub-programs' activities are subject to accountability.

Condition

Conclusion about the criterion:

The audit demonstrated that there is accountability for the sub-sub-programs' activities.

The sub-sub-programs' activities are subject to accountability.

We noted that there is an appropriate accountability for the sub-sub-programs' activities.

A monthly dashboard is prepared for the Director, Space Exploration Development as well as the Director General, Space Exploration. That information is also used to prepare the quarterly dashboard for all CSA projects that is presented to the President of the CSA. The dashboard includes information on the progress, recent highlights, issues, risks, scheduling, costs, and scope of projects.

Electronic work plans are also used as an accountability mechanism. The information contained in these work plans is focused on activity performance. Work plans are prepared annually and used in preparing the Departmental Results Report.

Recommendation

No recommendation.

Identified Responsibility
Organization

N/A

Function

N/A

Management Response

N/A

Management Action Plan
Action plan details

N/A

Deadline: N/A

Appendix A – Terms of reference

Audit objective: The objective of this audit was to determine whether a management framework is in place to enable sub-sub-programs' objectives to be achieved and to ensure compliance with relevant CSA and central agencies acts, policies, regulations and guidelines.,

Audit criteria Audit sub-criteria
  • Sub-criterion met 
  • Sub-criterion partially met 
  • Sub-criterion not met 
Criterion No. 1:
The sub-sub-programs' activities are planned appropriately.
Sub-criterion 1.1: Investment priorities are determined based on a complete analysis of opportunities. Sub-criterion met
Sub-criterion 1.2: Plans for financial resources have been made for planned activities. Sub-criterion met
Criterion No. 2:
The sub-sub-programs' activities are monitored appropriately.
Sub-criterion 2.1: Roles and responsibilities are clearly defined. Sub-criterion met
Sub-criterion 2.2: Projects are monitored and tracked as per the CSA's Investment Governance and Monitoring Framework and Project Management Policy. Sub-criterion met

Sub-criterion 2.3: Contractual agreements are awarded in accordance with the applicable guidelines, regulations, policies and legislation.

Furthermore, when required, contractual agreements are approved through the CSA's operational approval process for transfer payments and procurement.

Sub-criterion met

Sub-criterion 2.4: Grants and contributions are awarded as per the TBS Policy, Directive and Guideline on Transfer Payments, as well as the terms and conditions of the CSA's grant and contribution programs.

Agreements are also approved through the CSA's operational approval process for transfer payments and procurement.

Sub-criterion met
Sub-criterion 2.5: Spending authorizations are controlled, expenditures are approved and made in accordance with legislation and regulations, and transactions are recorded appropriately. Sub-criterion met
Criterion No. 3:
The sub-sub-programs' activities are subject to accountability.
Sub-criterion 3.1: The sub-sub-programs' activities are subject to accountability. Sub-criterion met
Date modified: