Evaluation of the Canadian Space Agency Space Expertise and Proficiency Sub-Program

For the period from to

Project # 16/17 – 02-01

Prepared by the Audit and Evaluation Directorate

Table of contents

  1. Introduction
  2. Background
  3. Evaluation approach and methods
  4. Evaluation findings
  5. Conclusions and recommendations

List of tables and figures

Acronyms used in the report

Executive Summary

This report presents the findings of the evaluation of the Space Expertise and Proficiency Sub-Program (hereafter referred to as the Sub-Program). The Sub-Program's fundamental purpose is to contribute to building and sustaining Canada's space capacity. It covers a range of activities that are expected to contribute to the development of highly qualified personnel (HQP) in space-related domains, and to increase knowledge in space science and technology (SST). During the evaluation period, the Sub-Program also pursued a broader goal of raising awareness and encouraging learning among Canadian youth and students in areas related to the space sector. The evaluation covers a five-year period, from to , and examines the Sub-Program's relevance, design and delivery, and performance.

The evaluation was conducted by PRA Inc., on behalf of the Canadian Space Agency (CSA)'s Audit and Evaluation Directorate, between and . The evaluation is a requirement of the CSA's five-year evaluation plan and was conducted in accordance with the Treasury Board of Canada's Policy on Results ().

Relevance

In terms of relevance, the Space Expertise and Proficiency Sub-Program is consistent with federal roles and responsibilities, and aligns with the CSA and federal priorities to develop HQP, advance SST knowledge, and promote science, technology, engineering, and mathematics (STEM) among youth and students. Overall, the evaluation indicates that there is a need for the Sub-Program to continue supporting the development of the Canadian space capacity. As such, the Sub-Program responds to the need of Canadian research institutions to engage in hands-on space projects. While other granting organizations play a critical and complementary role, what the Sub-Program offers has no equivalent in Canada. Moreover, having access to stratospheric balloons in Canada represents a well-suited tool to support SST and HQP development. The Sub-Program's activities contribute to the CSA's unique assets and attraction pole to engage young Canadians in STEM fields of study. Nonetheless, the evaluation identified opportunities for improvement for the Sub-Program to further respond to the needs of its beneficiaries and stakeholders.

Design and delivery

Overall, the Sub-Program components' design and delivery are sound and the CSA has established efficient structures and processes to manage it. The evaluation identified suggestions to strengthen the Sub-Program with respect to, among other things, the planning of research and engineering expertise activities, as well as the monitoring of the engineering expertise activities and the implementation of the Sub-Program's performance measurement.

Performance

During the period covered by the evaluation, the Sub-Program contributed to the establishment of a wide range of valuable partnerships involving universities and research centres, space industries, as well as other space agencies or space organizations. Activities under the Sub-Program have supported various forms of partnership, with a predominant number relating to end-to-end projects.

Sub-Program activities have also facilitated the involvement of students, both undergraduate and graduate, in a wide range of activities with a particular focus on hands-on projects. The evaluation findings have shown that these projects add a new and important dimension to the education students receive through their study programs. The Junior Engineers Training Program, managed under the Sub-Program, has also provided a unique opportunity for those selected participants to successfully engage in SST by joining the CSA workforce.

The Sub-Program has proven to be a key contributor, along with other granting organizations in Canada, to research projects performed in SST. These projects have supported the growth of SST knowledge in a number of areas targeted by the CSA. This has, in turn, contributed directly to the fundamental and legislative object of the CSA to "advance the knowledge of space through science."

In collaboration with the Centre national d'études spatiales (CNES), the Sub-Program has successfully launched stratospheric balloon campaigns throughout the period covered by the evaluation, thus providing access to pre-space and space environments. These campaigns have predominantly engaged students and academics, but also the space industry. Funding offered by the Sub-Program also provided researchers access to other demonstration opportunities, such as analog sites and parabolic flights.

Notwithstanding the achievement by the Sub-Program of its expected outcomes, the evaluation indicates that transitioning space-focussed students and HQP into the labour market remains a concern. While some of the students benefitting from the Sub-Program's activities will be able to pursue a space career in Canada in their field of expertise, evaluation findings indicate that others will need to apply generic skill sets in other space-related fields or apply these skills in other STEM-related fields that are not directly related to space. In this context, the implementation of the Sub-Program's activities should be done in such a way to support, when feasible, the transition of the trained HQP in various space-related fields, including space science and technology, Space Exploration, and Space Utilization, as well as in non-space fields.

In order to address the concern raised above, the evaluation recommends that

The Space Science and Technology Directorate ensures that its objectives and activities are aligned not only with current but also emerging needs of Canadian universities and research institutions in advancing space science and technology knowledge, while also facilitating the work transition of the developed HQP toward sectors, including but not limited to the Canadian space sector, where their advanced technical skills and behavioural competencies (soft skills) can be best applied.

Finally, the evaluation provided an opportunity to document how other activities in which the CSA is engaged contribute to the development of space-related HQP and SST. The goal was not to formally evaluate these activities, but rather to explore their relationship with the Sub-Program and their potential contribution to the achievement of the Sub-Program's expected outcomes. Along with the CSA Communications and Public Affairs Directorate's awareness activities, both the Space Exploration and Space Utilization branches engage universities and research institutions in a range of projects that, in a complementary manner, also further SST knowledge and train HQP.

1. Introduction

This document constitutes the final report of the evaluation of the Space Expertise and Proficiency Sub-Program (hereafter referred to as the Sub-Program). It covers a range of activities that are expected to contribute to the development of highly qualified personnel (HQP) in space-related domains, and to the increased knowledge in space science and technology (SST). The evaluation covers a five-year period, from to , and examines the relevance, design, and performance of the Sub-Program.

While the Sub-Program constitutes the fundamental scope of the evaluation, other activities undertaken by the Canadian Space Agency (CSA) in relation to the development of HQP and the increased knowledge in SST were considered to support a meaningful assessment of the Sub-Program. The goal was not to formally evaluate these activities, but rather to explore their relationship with the Sub-Program and their potential contribution to the achievement of the Sub-Program's expected outcomes.

The evaluation is a requirement of the CSA's five-year evaluation plan; it was conducted in accordance with the Treasury Board of Canada's Policy on Results ().

The evaluation was conducted by PRA Inc. on behalf of the CSA Audit and Evaluation Directorate, between and .

The contribution and collaboration of many individuals have made this evaluation possible. We wish to thank all of those who participated in data collection, provided information, and responded to inquiries.

2. Background

This section of the report includes a brief description of the Sub-Program and the broader context in which it operates. It covers the Sub-Program's key components, governance model, resource allocation, and expected outcomes. The purpose of this section is to provide sufficient contextual information to adequately assess the evaluation findings presented in the remaining sections of the report.

2.1 Key concepts

2.1.1 The development of HQP

In the context of this report and unless otherwise stated, HQP refers to post-secondary students and young researchers involved in space-related fields of study.

The report explores a number of dimensions related to the development of HQP and, consequently, it is helpful to clarify how HQP itself is to be understood in the specific context of this evaluation.

While frequently used, particularly in science, technology and innovation, the notion of HQP is not limited to a single generally accepted definition. It is context-driven, and typically serves to identify a sub-group of individuals who must possess more advanced qualifications to undertake certain tasks. For instance, Statistics Canada defines HQP "as individuals with university degrees at the bachelors' level and above" (Statistics Canada, ), while the CSA's definition of HQP focusses on engineers, scientists, and technicians (Canadian Space Agency, 2016f, p. 32). Not surprisingly, the workforce within the space sector includes a large proportion of individuals who fall within these broad definitions of HQP.

A more targeted definition of HQP focusses more specifically on post-secondary students and young researchers. At that stage, these individuals are essentially transitioning from having acquired advanced knowledge and competencies, toward positions that allow them to apply this knowledge as part of a time-bound project or by joining the space sector in the private sector, in government, or in academia. As an illustration, the Announcement of Opportunities for Flights and Fieldwork for the Advancement of Science and Technology (FAST) released by the CSA specifically defined HQP as including undergraduate students, graduate students (master's and PhD levels), and post-doctoral fellows (Canadian Space Agency, 2015b).

In the context of this evaluation, and considering the range of issues addressed, this more targeted definition of HQP is particularly helpful and, unless otherwise stated, it is the one that should be applied.

A final consideration in defining HQP relates to the field of work or study in which these individuals are involved or are in the process of getting involved. Just as it is the case with HQP, there is not a single and universally accepted definition of the space sector (OECD, , p. 8). The CSA has opted for the following definition: "The Canadian space sector is defined as organizations (private, public and academic) whose activities include the development and use of space assets and/or space data" (Canadian Space Agency, 2016f, p. 9). This broad definition is meant to include both the upstream segment (research, engineering, testing and consulting, and manufacturing) and the downstream segment (satellite operations, products and applications, and services) of the space sector (Canadian Space Agency, 2016f, p. 10).

As a result, the pool of space-related HQP includes both individuals whose qualifications directly relate to space science (e.g., astrophysics, planetary science, space-related life science), as well as individuals whose qualifications are not strictly limited to space, but find application in that field (e.g., computer engineering, electrical engineering, software developers, technicians).

2.1.2 Employment in the Canadian space sector

The Canadian space workforce includes 11,000 full-time equivalents (FTE) operating in the private sector, research institutions, as well as departments and agencies. A significant portion of theses FTEs falls within the CSA's broader definition of HQP that applies to individuals in the workforce (engineers, scientists, and technicians).

Considering how fluid the definitions of HQP, of the space sector, and of the space economy are, it will come as no surprise that some limitations apply when it comes to measuring the size of the space workforce. Moreover, as there are no specific North American Industry Classification System (NAICS) or National Occupational Classification (NOC) codes that may be used to isolate space-specific employment, we must rely on surveys conducted by the CSA or other organizations to provide an overall estimate of that workforce (Hickling Arthurs Low, , p. 5).

On that basis, and considered in its broader sense (upstream and downstream segments, HQP and non-HQP), the Canadian space workforce is estimated to include approximately 11,000 FTEs:

  • Over 10,000 FTEs are employed in private-sector enterprises or in universities and research centres involved in the space economy (Canadian Space Agency, 2016f, p. 30).
  • Roughly 1,000 FTEs are involved in space-related employment with the CSA, the Canadian Ice Service, the Canadian Meteorological Service, and the Centre for Remote Sensing (Hickling Arthurs Low, , p. 44). On its own, the CSA engages close to 620 FTEs (including both HQP and non-HQP) (Canadian Space Agency, 2016a, p. 6).

The space workforce includes a variety of employment categories. Among them, engineers, scientists, and techniciansFootnote 1 are considered as HQP as per the CSA's definition (Canadian Space Agency, 2016f, p. 32). As illustrated in Figure 1, these categories represented 42% (or 4,226 FTEs) of the space workforce (excluding government employees) in .

Figure 1: Space workforce distribution, excluding government employees (2014)
Administration Engineers &
Scientists
Technicians Other Management Students
& Interns
Marketing
and Sales
Distribution 34% 31% 11% 8% 7% 5% 4%

(Source: State of the Canadian Space Sector )

In academia, there are approximately 50 research institutions and universities that are involved in space-related studies in Canada (Euroconsult, , p. 57). As of , they were employing roughly 1,930 FTEs, of which 71% were within the HQP category when using the CSA's definition, which includes engineers, scientists, and technicians (Canadian Space Agency, 2016f, p. 13).

2.2 Description of the Sub-Program

2.2.1 Overview

The Sub-Program's fundamental purpose is to contribute to building and sustaining Canada's space capacity. More specifically, it "supports research in private or public organizations and sustains the development of highly qualified personnel (HQP) in science and engineering" (Canadian Space Agency, 2016e, p. 11).

Scientists and engineers who benefit from the support provided by the Sub-Program are expected to undertake relevant developmental activities related to SST. This is achieved mainly through direct support to research activities, along with access to infrastructure devoted to space-related research, including flight opportunities through fast execution and small-size missions.

The Sub-Program also pursues a broader goal of raising awareness and encouraging learning among Canadian youth and students in areas related to the space sector. While still active to some degree (e.g. by providing funding support to graduate students to help them participate in space conferences or similar events), it should be noted that most of these activities have been suspended during the period covered by this evaluation, following the CSA's program review.

2.2.2 Components

Five components are included under the Sub-Program and are described in the following sub-sections.

2.2.2.1 Science and Academic Development (SAD)

The primary purpose of the SADFootnote 2 component is to offer financial support to eligible individuals or organizations in order for them to undertake research activities that will both enhance the skill set of students involved in space science and technology research, and address research areas that are relevant to the CSA.Footnote 3

During the period covered by the evaluation, the support provided by the SAD component focussed predominantly on four types of research activities (Canadian Space Agency, 2014a, p. 74):

  • Flights and Fieldwork for the Advancement of Science & Technology (FAST), which supports Canadian university researchers that offer hands-on experience in space-like missions to students and facilitate the participation of researchers (including post-doctoral fellows and research associates);
  • academic development activities, which support research and development clusters of three or more principal investigators from three or more organizations (universities, not-for-profit organizations and/or private organizations);Footnote 4
  • collaborative science research, which supports collaboration between CSA employees and other organizations; and
  • research infrastructure activities, which support university research and training university projects using different research facilities.

As the description of its activities suggests, the SAD component provides funding to eligible recipients, which are post-secondary institutions (mostly universities). The funding takes the form of either grants or contributions.

In order to allocate and manage the funding provided through the SAD component, the CSA uses its Class Grant and Contribution Program (Class G&C Program). The overall goal of the Class G&C Program is to support "knowledge development and innovation in the CSA's priority areas while increasing the awareness and participation of Canadians in space-related disciplines and activities" (Canadian Space Agency, 2016g, p. 6). The Class G&C Program has two components, the Research component, under which fall the SAD grants and contributions, and the Awareness and Learning component. Using the Class G&C Program, the SAD component may provide funding through solicited or unsolicited proposals. According to the terms and conditions of the Class G&C Program (Canadian Space Agency, 2016b):

  • "Solicited proposals are submitted in response to a CSA, time-bound, Announcement of Opportunity (AO) to address certain themes and/or sectors of intervention and/or certain types of recipients. Projects are evaluated against the [Class G&C] Program's assessment criteria and in comparison with each other. Funds are set aside for this purpose."
  • As for unsolicited proposals, "proponents may submit, at any time, unsolicited proposals in line with CSA's priorities. Projects are evaluated individually against the [Class G&C] Program's assessment criteria. They may be approved based on merit and availability of funds."

Whether it is granted through solicited or unsolicited grants or contributions, any funding provided through the SAD component must (in addition to the criteria associated to each of the four types of research activities) respond to at least one of the following objectives identified in the Class G&C Program (Canadian Space Agency, 2016b):

  • to support the development of science and technology relevant to the priorities of the CSA;
  • to foster the continuing development of a critical mass of researchers and HQP in Canada in areas relevant to the priorities of the CSA;
  • to support information-gathering, studies, and research related to space; and
  • to support the operations of organizations dedicated to space research.Footnote 5

The following table provides an overview of the funding provided through the SAD component during the five years covered by the evaluation.

Table 1: Actual expenditures of the Science and Academic Development (SAD) component ($)
Funding categories Total
Salaries 1,573,886 1,334,830 423,214 302,181 191,483 3,825,594
O&M 1,740,101 340,376 105,403 295,269 174,096 2,655,242
Grants 6,090,016 3,556,250 3,542,489 3,619,699 2,857,031 19,665,484
Contributions 2,391,055 535,212 51,000 0 144,506 3,121,773
Total 11,795,058 5,766,667 4,122,106 4,217,148 3,367,113 29,268,093

Source: CSA financial data, .

At $29.3 million over five years, the SAD component represents approximately half of all the expenditures that fall under the Sub-Program. The bulk of the expenditures under the SAD component were directed toward grants or contributions, which were provided to universities. The operations and maintenance category typically covers expenditures related to access to research facilities.

2.2.2.2 Engineering Professional Development (ED1)

The Engineering Development Directorate, under the Space Science and Technology branch, includes a team of engineers and other professionals who undertake assignments and projects related to various dimensions of space science and technology, including flight software, power and digital electronics, spacecraft payload laboratory, optical sensors, material and thermal, control and analysis, structures, and system engineering.

A portion of the work undertaken by this team of professionals is assigned to specific projects of the CSA where they may act as project or technical authority. This work is specifically excluded from the evaluation scope, as it is covered under other CSA sub-programs.

The other portion of the work undertaken by the Engineering Development Directorate team members covers a range of activities that falls under the ED1 component covered by this evaluation. These activities include (but are not limited to):

  • supporting initiatives undertaken by external partners (universities, other federal departments, or private partners);
  • assisting CSA corporate sectors (such as communications or policy-related activities);
  • participating in investigations to resolve spacecraft anomalies;
  • responding to unplanned technical issues;
  • presenting at conferences and other professional development fora;
  • developing frameworks for future programs or initiatives;Footnote 6 and
  • teaching postgraduate courses, or participating in co-op student placements.Footnote 7

Funding allocated to ED1 is also used to purchase and maintain specialized software licences, and to maintain various laboratories owned by the CSA.

All resources invested in ED1 are internal to the CSA and include salaries, operation and maintenance (O&M), and capital expenditures (for laboratories), as described in the following table.

Table 2: Actual expenditures of the Engineering Professional Development (ED1) component ($)
Funding categories Total
Salaries 2,000,476 2,400,000 1,546,062 1,015,970 260,553 7,598,892
O&M 1,762,314 1,279,823 1,281,596 971,735 788,609 6,084,078
Capital 256,472 89,279 104,614 148,983 0 599,347
Total 4,023,262 3,769,812 2,932,271 2,136,688 1,049,162 14,282,317

Source: CSA financial data, .

At $14.3 million over five years, ED1 has mobilized 25% of all resources invested in the Sub-Program. It is important to emphasize that all the work undertaken by engineers and other professionals within the Engineering Development Directorate that is directly assigned to a CSA project is excluded from the budget presented in Table 2.

2.2.2.3 STRATOS

STRATOS is the CSA's stratospheric balloon program. The initiative was established in through a collaboration agreement signed between the CSA and the Centre national d'études spatiales (CNES) of France (Canadian Space Agency, 2013b). It became operational in . This program gives Canadian academia and industry the opportunity to test and validate new technologies and to perform scientific experiments in a near-space environment. In doing so, STRATOS contributes to the scientific and technological advancement of the Canadian space sector by supporting demonstration activities, which contribute to developing current and future Canadian engineers and scientists.

STRATOS operates through balloon flight campaigns, each of which may last six to eight weeks. Between four and 10 balloon launches are executed per campaign, each carrying a number of payloads designed and developed by engineers and scientists from various countries, including Canada. Balloons are launched from a number of locations around the world, including the Stratospheric Balloon Base that was built in Timmins, Ontario, in (Canadian Space Agency, 2013c).In addition, and for the first time in , the STRATOS program directly involved high school students, allowing their experiment (the Lightning Probe) to be placed on a flight launched from Sweden in (Canadian Space Agency, 2016c).

The STRATOS program provides the technical assistance associated with the actual balloon flights. As such, it does not provide any direct assistance for the development of the technology to be placed on a stratospheric balloon. This type of assistance can be obtained through other research and development initiatives offered by the CSA, including FAST.

The largest portion of the STRATOS budget is allocated to operation and maintenance, which includes lease, rental, professional services, travel, and training fees.

Table 3: Actual expenditures of the STRATOS component ($)
Funding categories Total
Salaries n/a 350,478 1,090,304 1,021,184 712,959 3,174,925
O&M n/a 1,009,556 2,757,680 1,089,117 929,740 5,786,093
Capital n/a 0 130,086 38,184 0 168,271
Total n/a 1,360,034 3,978,070 2,148,485 1,642,699 9,129,289

Source: CSA financial data, .

2.2.2.4 Junior Engineers Training Program

In order to support the integration of new engineers within the CSA, in the Engineering Development Directorate launched the Junior Engineers Training Program. Through this initiative, a number of new engineers are brought to the CSA and, while they are assigned to various branches within the agency, their salary is covered by the program for a two-year period. The goal is to have these other branches fully integrate the new recruits (including their salaries) after the two-year period.

During the first round, in , the CSA hired eight new recruits. During the second round, in , the CSA hired four new recruits through the program. The budget of this program largely covers the salaries of the new engineers.

Table 4: Actual expenditures of the Junior Engineers Training component ($)
Funding categories Total
Salaries 506,154 467,814 67,646 0 47,172 1,088,787
O&M 39,293 7,505 4,750 1,180 80 52,808
Total 545,446 475,320 72,396 1,180 47,252 1,141,595

Source: CSA financial data, .

2.2.2.5 Awareness and Learning and Space Learning components

The last two components of the Sub-Program focus on Canadian students and aim to raise their awareness of space-related disciplines and to provide learning opportunities in these fields.

The Class G&C Awareness and Learning component

The Class G&C Program (briefly described in Sub-section 2.2.2.1 of this report) includes an Awareness and Learning component that was suspended by the CSA in following a program review (Canadian Space Agency, 2013a). While active, the primary goal of this component was to provide "financial support to individuals and organizations involved in initiatives targeting Canadian youth, students, physicians and educators to increase their awareness, knowledge, development and participation in space-related disciplines and/or activities and advanced educational programs" (Canadian Space Agency, 2016b). As an illustration, this component funded curriculum-based education activities identified through consultations with provincial education boards. It also supported Space Learning activities by, for instance, funding travel expenses incurred by recipients to participate in conferences or other space learning events.

The CSA continues to provide funding support to graduate students to help them participate in space conferences or similar events, but this funding is currently provided through the SAD component.

Space Learning component

The Space Learning component of the Sub-Program focussed on the development and implementation of education-related activities targeting Canadian youth. It allowed the CSA to participate in space-related events or activities, or make curriculum-based presentations that exposed Canadian youth to various dimensions of space activities. As detailed in Table 5, all direct expenditures allocated to the Space Learning component of the Sub-Program were composed of salaries and O&M; however, as mentioned above, the Space Learning component made use of the Awareness & Learning component of the Class G&C Program to fund portions of its activities.

Since the program review, these activities are no longer supported through the Sub-Program. As a result, no expenditures have been identified under the Space Learning component of the Sub-Program since . Of note, the CSA's Communications Function has maintained its own awareness activities (participation in exhibits and events, web updates, social media communications, etc.) in accordance with its mandate.

As described in Table 5, the CSA invested a total of $2.7 million over the period covered by the evaluation to support these two components of the Sub-Program.

Table 5: Actual expenditures for Awareness and Learning components ($)
Funding categories Total
Awareness and Learning (Class G&C Program)
Grants 278,000 10,000 0 0 0 288,000
Contributions 948,200 169,209 0 1,785 0 1,119,194
- Sub-total 1,226,200 179,209 0 1,785 0 1,407,194
Space Learning
Salaries 331,558 65,987 0 0 0 397,545
O&M 678,375 172,987 0 0 0 851,362
- Sub-total 1,009,933 238,974 0 0 0 1,248,907
Total 2,236,133 418,183 0 1,785 0 2,656,101

Source: CSA financial data, .

2.2.3 Resources and management

2.2.3.1 Overall budget

Table 6 includes a summary of the total investment that the CSA made in the Sub-Program during the five years of the period covered by the evaluation. As illustrated in the table, the SAD and ED1 components combined represent close to 80% of the total investment made in the Sub-Program.

Table 6: Actual expenditures for the Sub-Program ($)
Total
SAD 11,795,058 5,766,667 4,122,106 4,217,148 3,367,113 29,268,093
ED1 4,023,262 3,769,812 2,932,271 2,136,688 1,049,162 14,282,317
STRATOS 0 1,360,034 3,978,070 2,148,485 1,642,699 9,129,289
Junior Engineers Training Program 545,446 475,320 72,396 1,180 47,252 1,141,595
Space Learning 1,009,933 238,974 0 0 0 1,248,908
Awareness and Learning 1,226,200 179,209 0 1,785 0 1,407,194
Total 18,599,899 11,790,016 11,104,843 8,505,286 6,106,226 56,477,396

Source: CSA financial data, .

2.2.3.2 Full-time equivalent (FTE)

Table 7 includes the number of FTEs assigned to the Sub-Program. It indicates that the level of staff assigned to the Sub-Program decreased significantly in , as a result of the measured adopted in the CSA's program review. In particular, the reduction in FTEs assigned to SAD resulted mainly from the dissolution of the Space Science Directorate, while the reduction in FTEs assigned to ED1 is mainly attributed to the shift of FTEs from ED1 to STRATOS. In addition, Space Learning FTEs were used to administer the G&Cs allocated under the Awareness and Learning component.

Table 7: Full-time equivalents assigned to the Sub-Program
Total
SAD 16.18 12.79 5.05 2.63 1.80 38.45
ED1 21.76 24.34 9.29 7.05 5.68 68.12
STRATOS 0 2.89 8.77 9.99 6.57 28.22
Junior Engineers Training Program 8.05 6.79 1.04 0 0.58 16.46
Space Learning 5.08 1.09 0 0 0 6.17
Awareness and Learning 0 0 0 0 0 0
Total 51.07 47.90 24.15 19.67 14.63 157.42

Source: CSA financial data, .

2.2.3.3 Roles and responsibilities

The CSA includes three programmatic branches, namely, Space Utilization (SU), Space Exploration (SE), and SST. The Director General of the SST branch is responsible for the overall management of all components of the Sub-Program. He is supported by the director of the Engineering Development Directorate and the director of the Technology Development Management Directorate, who are responsible for various components, as indicated in Figure 2.

Figure 2: Sub-Program Management Structure

Sub-Program Management Structure Graph. Text version below:
Figure 2: Sub-Program Management Structure - Text version

Space Science and Technology Branch divided in 2 branches: Engineering Development Directorate and Technology Development Management Directorate.

  • Under Engineering Development Directorate:
    • Engineering Professional Development (ED1)
    • STRATOS Program
    • Junior Engineers Training Program
  • Under Technology Development Management Directorate:
    • Science and Academic Development (SAD)

To facilitate the ongoing management of the Sub-Program, the CSA has established a number of tools, including an updated Project Management Policy, which provides guidance related to the planning, approval, and development of new projects, as well as a policy on investment planning related to investments in assets and services (Canadian Space Agency, 2016e).

As previously noted, the CSA also uses its Class G&C Program to support the selection and management of projects for which grants or contributions are allocated.

2.2.4 Program logic

The logic of the Sub-Program is described in its Performance Measurement Strategy (Canadian Space Agency, 2016e).

Figure 3: Logic Model – Space Expertise & Proficiency (1.3.1)

Logic Model – Space Expertise & Proficiency (1.3.1). Text version below:
Figure 3: Logic Model – Space Expertise & Proficiency (1.3.1) - Text version

Inputs:

  • Funding as per SAP/IPS – FTE – Consultations with stakeholders

Activities:

  • Providing space science and technology (SST) training opportunities to students, HQP and workforce (A1)
  • Providing demonstration opportunities (A2)
  • Providing access to expertise, laboratories and infrastructures (CSA internal and external) (A3)

Outputs:

  • Agreements signed, internships for students and HQP and training events (Op1)
  • Conducted Demonstrations (Op2)
  • Availability of laboratories and infrastructures (Op3)
  • CSA Expert advice and support provided to organizations pursuing space activities (Op4)

Outcomes:

Immediate:

  • Partnerships amongst research organizations established or maintained (Oc1)
  • Increased or maintained number of students and HQP in SST (Oc2)
  • Increased or maintained number of research projects performed in SST (Oc3)
  • Increased or maintained possibilities or accessing pre-space and space (OC4)

Intermediate:

  • Sustained HQP in the space sector (Oc5)
  • Increased knowledge in SST (Oc6)

Ultimate:

  • Sustainable Canadian capabilities contribute to space opportunities (Oc7)

2.3 Prior and related evaluations of the Sub-Program

This is the first time that the Sub-Program is being evaluated (Canadian Space Agency, 2016e, p. 10). However, a recent evaluation of the CSA Class G&C Program provides relevant insights (Canadian Space Agency, 2017b). While the scope of that evaluation reaches beyond the Sub-Program,Footnote 8 it overlaps with the SAD component, which is using the Class G&C Program to manage the grants and contributions it provides (as noted in sub-section 2.2.2.1).

Looking at the delivery of grants and contributions over a five-year period ( to ), the report indicates that "effective funding applications mechanisms were in place, the AO selection process was fair, and the reporting requirements were similar to those of other federal departments" (Canadian Space Agency, 2017b, p. 80). One challenge associated with the delivery of the Class G&C Program is around the unpredictability of AOs. On this point, the report notes that "the infrequency and unpredictability of AOs pose challenges for the Canadian space community in terms of planning for and maintaining HQP and infrastructure capabilities" (Canadian Space Agency, 2017b, p. 38).

The evaluation report concludes that, overall, the Class G&C Program has contributed to the achievement of its expected outcomes. In particular, the Class G&C Program is increasing SST knowledge and Canada's focus on space, and it is fostering collaborations between funding recipients and both national and international partners.

On the issue of outreach to Canadian youth to promote STEM and space, which was done through the Awareness and Learning component of the Class G&C Program until it ceased funding activities aimed at elementary and secondary school students as part of the program review measures, the report concludes that "the alignment between the Awareness and Learning component and the CSA's priorities is tenuous" (Canadian Space Agency, 2017b, p. 31). This led to a recommendation that the "Class G&C Program's Terms and Conditions should be reviewed to determine whether the Awareness and Learning component remains aligned with the CSA's priorities" (Canadian Space Agency, 2017b, p. 82).

3. Evaluation approach and methods

This section of the report provides a brief description of the methodology used to conduct the evaluation of the Sub-Program. It clarifies the purpose and scope of the evaluation, describes the key evaluation issues being addressed, and the methods used to gather evaluation findings. It also identifies the limitations that the evaluation faced, along with the strategies used to mitigate these limitations.

3.1 Purpose and scope

This report fulfills the commitment included in the CSA's Departmental Evaluation Plan ( to ) to conduct the evaluation of the Space Expertise and Proficiency Sub-Program (Canadian Space Agency, 2015a). It covers a five-year period, from to .

While the Sub-Program constitutes the fundamental scope of the evaluation, it was recognized that other activities undertaken by the CSA in relation to the development of HQP needed to be considered to support a meaningful assessment of the Sub-Program. For this reason, certain data collection activities reached beyond the Sub-Program to include, for instance, activities undertaken by the Space Utilization and Space Exploration branches, or the Communications Directorate. In these latter cases, the goal was not to formally evaluate these activities, but rather to explore their relationship with the Sub-Program and their potential contribution to the achievement of the Sub-Program's expected outcomes.

The evaluation covers the relevance, design, and performance of the Sub-Program. The evaluation questions are as follows:

Relevance
  • To what extent does the Sub-Program continue to address a demonstrable need and respond to the needs of Canadians?
  • How is the Sub-Program aligned with the CSA's goals and priorities related to the development of highly qualified personnel in space science and technology?
  • How is the Sub-Program aligned with the Government of Canada's agenda related to space activities?
  • To what extent do activities undertaken through the Sub-Program reflect an appropriate distribution of roles and responsibilities related to the development of highly qualified personnel in space science and technology?
Program design
  • How efficient is the Sub-Program's delivery structure?
  • Is the Sub-Program supported through an adequate performance measurement strategy?
Performance (effectiveness)
  • To what extent has the Sub-Program achieved its immediate outcomes?
  • To what extent has the Sub-Program achieved its intermediate outcomes?
  • To what extent has the Sub-Program contributed to the achievement of its ultimate outcome?
  • Have any unexpected outcomes resulted from the activities undertaken through the Sub-Program?
Performance (efficiency and economy)
  • What measures has the CSA implemented to optimize the use of the Sub-Program's resources?

3.2 Methods

Evaluation data were collected through a number of research methods, which are briefly described in this sub-section.

3.2.1.1 Document and data review

This review was initiated by an analysis of documents and information pertaining to each component of the Sub-Program. This included relevant CSA policies and strategies, planning documents, program descriptions, and other departmental documents (including performance reports, presentations, or background documents).

The second step consisted of reviewing publicly available documents and publications covering more broadly issues such as space workforce and employment, participation in and promotion of STEM, particularly among youth, HQP development, and gender representation among HQP and the space workforce. This review covered the Canadian experience, as well as the experience of other nations involved in space activities.

The third step consisted of a review of relevant administrative data related to each Sub-Program component, as applicable. In particular, the information through the CSA UNITAS database provided detailed insights regarding the range of projects funded through the Sub-Program, including an assessment of project results. The data review also included, to the extent possible, internal activities such as those associated with the engineering development support (ED1), STRATOS, or the Junior Engineers Training Program.

Finally, the document and data review was used to inform each of the case studies.

3.2.1.2 Key informant interviews

Key informant interviews contributed to the in-depth understanding of the Sub-Program, including results achieved and challenges faced by each of its components. These interviews also corroborated, explained, or further elaborated on findings from other data sources and provided important input into whether outcomes have or have not been achieved, and why they have or have not been achieved.

A total of 52 interviews of approximately one hour each were conducted, which involved 61 individuals (some were group interviews) from seven different stakeholder groups. The distribution of these interviews is included in Table 8.

Table 8: Distribution of interviews conducted as part of the evaluation
Key informant groups # of interviews # of individuals interviewed
CSA representatives 14 23
Academics (funded and non-funded) 18 18
STRATOS (program participants – including industry representatives – and CNES representatives) 10 10
Students and interns 5 5
Other industry representative 1 1
Other federal departments 2 2
Other space agencies 2 2
Total 52 61

3.2.1.3 Case studies

A total of six case studies were completed as part of this evaluation. The list of these case studies includes both activities contained within the Sub-Program as well as other activities that fall beyond the Sub-Program, but whose relevance and expected results directly relate to the goals that the Sub-Program pursues. More specifically, the list of case studies includes:

  • Mars Sample Return Simulations;
  • Tomatosphere;
  • the Canadian Satellite Design Challenge;
  • the Junior Engineers Training Program;
  • the "End-to-End Airborne Quantum Key Distribution (QKD) Demonstration" project funded under SAD; and
  • the "Satellite prototype development of the Aerosol Limb Imager (ALI) instrument" project supported through STRATOS and SAD.

Each case study included up to four interviews with stakeholders associated with the project, program, or activity, including external stakeholders (recipients, participants, industry representatives, etc.) and internal CSA stakeholders. In total, 16 interviews were conducted involving 29 individuals.

Case studies also involved a review of project, program, or activity-level documents and data maintained by the CSA, as well as publicly available information (e.g., websites, online documents). Short case study reports were prepared using a standard case study report template.

3.3 Limitations

A number of data limitations related to the evaluation of the Sub-Program had to be addressed.

Performance measurement strategy development

The Sub-Program's performance measurement strategy was developed and approved at the onset of the evaluation. Consequently, the data collected to date address a number but not all associated performance indicators. The strategy also provides an updated description of the Sub-Program and includes relevant information regarding how the Sub-Program intends to collect data and the nature of the data to be collected.

Changes to the Sub-Program

The actual program or activities evaluated has undergone substantial changes since . During the evaluation period, activities under two components of the Sub-Program (Awareness and Learning and Space Learning) have been substantially modified. As a result, some of the program data and the access to former program staff were limited. The document review therefore played a crucial role in the assessment of this component.

Evaluating a program that has never been evaluated

Since the Sub-Program had never been evaluated, the preliminary consultations and the initial review of documents and databases played a particularly critical role in order to appropriately orient the evaluation matrix, data collection activities, and analysis. The expansion of the scope of the evaluation also served to appropriately contextualize the activities undertaken through the Sub-Program.

Burden on potential key informants

At the time that this evaluation was undertaken, the CSA was completing the Class G&C Program evaluation. In addition, the CSA had also initiated consultations in relation to the federal government's Innovation and Skills Plan. In this context, it became particularly important to manage interviews in such a way as to avoid overburdening key informants. To this end, careful considerations were given to the input already provided through these other related initiatives. The data collection process conducted as part of the evaluation ensured that complementary information was gathered wherever it was required to adequately address the evaluation issues and questions identified in the evaluation matrix.

Most interviewees have a vested interest in the Sub-Program

This limitation was mitigated by requiring interviewees to explain their perspectives and provide examples where appropriate. In terms of the overall report, the findings from the key informant interviews were triangulated with findings from other data sources (document review, administrative data, financial data, and case studies).

4. Evaluation findings

This section of the report describes the evaluation findings. When appropriate, findings on the Sub-Program as a whole are presented, but to enhance the utility of the report, findings are predominantly presented by component of the Sub-Program, covering first their relevance and then their performance. Broader contextual information is also provided to support the analysis. These considerations have directly informed the structure of this section of the report, and should be kept in mind during its review.

The information is based on findings that emerged from all lines of evidence. Unless otherwise noted, when opinions are reported, these are the opinions of the stakeholders consulted, and not those of the evaluators.

4.1 Relevance

For the purpose of this report, the assessment of the Sub-Program's relevance is informed by the extent to which it has been addressing a demonstrable need, it is still aligned with the priorities of the CSA and of the federal government more generally, and it is reflecting an appropriate distribution of roles and responsibilities.

4.1.1 Addressing a demonstrable need

Research institutions play an important role in advancing SST knowledge and in training HQP. To this end, they require the ability to undertake hands-on research projects, such as those funded through the Sub-Program or through other activities of the CSA. Well-rounded HQP with space-related expertise can, in turn, engage in the space private sector, or in other organizations such as federal departments and agencies using space data, recognizing that challenges do exist for those interested in building a career in space.

It is also widely acknowledged that the development of future HQP is a broad endeavour that starts in the early years of education, through engaging activities promoting STEM.

Developing HQP and promoting STEM are fairly broad goals and, as such, it is helpful to further clarify how the need for such activities has been documented and how it is perceived by key stakeholders. The following sub-sections summarize applicable evaluation findings before turning to each component of the Sub-Program and to other contributors that must be considered.

4.1.1.1 Developing HQP and facilitating SST knowledge growth
The involvement of Canada in space

As with all open economies, Canada must continuously support, grow, and renew its pool of HQP in order to maintain its privileged position in various markets. In launching its Innovation and Skills Plan in , the federal government emphasized the role of innovation, fuelled by highly skilled workers, researchers, and entrepreneurs, in fostering economic growth. It noted that "entire industries are being transformed as markets and businesses race to adapt" and that, while Canada continues to evolve, "other countries are moving even faster to equip their economies to lead" (Government of Canada, , p. 3).

With total revenues of roughly $5.5 billion, the upstream and downstream segments of the space sector contribute significantly to Canada's innovative economy (Canadian Space Agency, 2016f, p. 12). This achievement rests on several building blocks that include a fairly extensive network of approximately 50 research institutions and universities engaged in space-related studies in Canada (Euroconsult, , p. 57), in addition to a multitude of academic institutions that provide Canadians with the opportunity to acquire other, more generic, skills and competencies needed to support the space sector.

It is worth recalling that the country's level of involvement in space has historically exceeded its relative demographic size. By being the third country (after Russia and the United States) to launch its own satellite (Canadian Space Agency, 2012a), Canada engaged in a path marked by striking innovation (e.g., Canadarm) and the participation of eight astronauts in 16 space missions to date (Canadian Space Agency, 2016k). As emphasized during interviews conducted as part of this evaluation, just a handful of countries possess a space heritage comparable to Canada's. But, if the space-faring field has long been occupied by a selected few, such is no longer the case. A recent study commissioned by the CSA notes that the number of countries investing significantly in space has increased from 15 in to 58 in . Civil investments in space by Japan, India, Germany, Italy, the United Kingdom, and Brazil now exceed Canada's (Euroconsult, , pp. 4–5).

The strategic choice made by Canada to remain actively involved in space comes with a series of requirements in terms of planned missions, industrial capacity, and scientific development. While all three dimensions are intertwined, this evaluation is largely focussed on the latter two.Footnote 9 Having a solid network of research institutions and universities capable of training space-focussed scientists and workers is a fundamental precondition for Canada to maintain its space-faring status. It is also a goal that is directly reflected in the Canadian Space Agency Act, which states that one of the objects of the CSA is "to advance the knowledge of space through science" (Statutes of Canada, , sec. 4).

It is within this larger context that the evaluation explored the range of needs and challenges that key stakeholders face when it comes to developing HQP and growing SST knowledge.

Needs of research institutions and universities

The range of interviews conducted as part of this evaluation provided an opportunity to better understand how research institutions and universities can best support those students interested in space-related disciplines. The goal of the evaluation is not to assess the overall quality of the education offered by Canadian universities. Rather, it is to better understand how funding, such as that offered by the CSA (in conjunction with other granting institutions in Canada, as applicable), can enhance the education offered to Canadian students, particularly those involved in space-related fields of study. Results from consultations with academia held by the CSA in in relation to the Innovation and Skills Plan were also considered, as they echoed many of the interview findings while providing additional insights (Canadian Space Agency, 2016d).

Support to academic teaching

In addition to furthering scientific knowledge, research projects undertaken by faculty members are directly included in the academic teaching they offer to students. Evaluation findings indicate that, in the case of undergraduate teaching, research projects are integrated in course content in order to provide a variety of examples and illustrations of the nature and range of results obtained through these projects. When feasible and appropriate, undergraduate students are also directly involved in the execution of research projects.

In the case of graduate students, research projects are at the core of the education they receive. These students typically engage in the projects themselves, which may include the development of funding submissions, as well as the execution of the research. As one would expect, the level of involvement varies between master's and PhD students, but the primary goal is to engage these students in completing the research, which often becomes part of a student's thesis, in addition to opening-up other opportunities for publishing in scientific journals or presenting at conferences.

In this context, a primary responsibility of faculty members is to secure ongoing funding in order to be in a position to provide these hands-on research opportunities to students. Evaluation findings illustrated how challenging this task may prove to be. By their very nature, space-related research projects tend to be long-term, which may extend several years. However, research funding is normally structured in cycles of three to five years. Examples of projects that nearly died when trapped between funding cycles were provided by those consulted as part of this evaluation. According to key informants, this challenge is also more acutely felt by new faculty members, who often lack the capacity to successfully compete for funding and build a stable funding environment in which they can engage their graduate students. As noted during interviews, the reduced funding opportunities available to post-doctoral fellows further accentuate this challenge.

Range of activities funded

In terms of the range of activities funded through research projects, the following needs emerged from the evaluation:

  • End-to-end projects offer many advantages and remain an important avenue to support a wide range of knowledge and skill development.
  • The design of new instruments is more challenging to undertake, as current funding streams for this type of research are more restricted.
  • Another area where securing funding is proving challenging is data analysis and, more generally, scientific research, since funding has tended to focus on demonstration activities and earlier stages of research more generally.
  • Access to laboratories and research facilities provides critical training opportunities, but interview findings indicate that it has also proven to be more challenging to fund in recent years.
  • To adequately bridge graduate students with industry as part of research projects remains a concern among key informants interviewed. It is seen as being particularly critical when considering, as per the key informants' perception, the limited space-related employment openings available in other areas of the space sector, such as academia and governments.
  • A challenge faced more specifically by young faculty members is the ability to secure "middle-range" funding (smaller research projects) that may allow them to build their experience and provide a bridge toward more traditional and larger research endeavours.

Participation of women in space studies

Engaging women in STEM is a long-standing challenge that many countries face, and Canada is no exception (Department of Finance Canada, ). Compared to men, a greater proportion of woman pursue post-secondary education, but they are more likely to do so in the fields of arts, humanities, or social sciences (Hango, 2013a, p. 19), as opposed to STEM-related fields. More recent trends, however, point toward a greater gender balance in STEM. For instance, in , "39% of STEM university graduates aged 25 to 34 were female, compared to 23% of STEM graduates aged 55 to 64" (Hango, 2013b, p. 2).

When examining more specifically the various fields included within STEM, it must be noted that in women represented only 23% of all graduates in engineering and 30% of all graduates in mathematics or computer science (Hango, 2013b, p. 2). As one could expect, a lower representation of women among graduate studies also contributes to a less visible presence of female faculty in STEM university programs (Stauffer, ).

Interviews with faculty members conducted as part of this evaluation largely echoed these observations. Despite the fact that STEM studies tend to be dominated by male students, it was noted that a greater representation of female students is found in a number of STEM programs. Also, while engineering and physics studies are still being perceived as being quite unbalanced, a greater balance is reported by key informants in fields such as biology and chemistry. In the case of medicine and health science, perceptions reported during interviews are that women now constitute the majority of students.

Needs of the space industry

As noted in sub-section 2.1.2 of this report, the space sector offers a wide range of employment opportunities, some of which require space-specific knowledge, while others require more generic skillsets that may have little to do with space itself. For the purpose of this evaluation, it was particularly relevant to explore the extent to which the space industry is capable of filling those positions that require the type of knowledge and competencies that are offered by space-oriented academic programs in Canada, as these HQP are the specific focus of the Sub-Program.

Considering more generally the expected demand for labour in STEM-related fields (including space and non-space fields), it must first be acknowledged that these areas of employment, particularly when it comes to aerospace and space, have historically been highly cyclical (John O'Grady Consulting Ltd., , p. 11). At the time of this evaluation, studies were indicating that, while short-term demand for STEM-trained employees were being met, shortages were expected over the next decade (NSERC, , p. 6), which may increase competition among high technology sectors for recruiting qualified personnel, as previously noted.

These findings were corroborated by industry representatives consulted as part of this evaluation (during both key informant interviews and case studies). They emphasized the cyclical nature of the space sector and the challenges that space industries face in forecasting future demands from the CSA or from foreign agencies or other entities, something that has a direct impact on workforce recruitment (see also Aerospace Industries Association of Canada, , p. 13).

Interview findings also echoed the notion that, in private-sector companies, the vast majority of employees are recruited at the bachelor level, typically from one of the engineering disciplines. There is certainly a need for graduate students in space science-related disciplines, but, as one key informant put it, "for every graduate researcher you hire, you need 20 engineers to actually create the products."

Interview findings highlighted a particular challenge faced by the space (and aerospace) industry when it comes to recruiting students. As a result of the requirements under the Controlled Goods Program, which covers goods that have military or national security significance, there are a number of students in Canadian universities who cannot be recruited, as they cannot obtain the required security clearance (Public Services and Procurement Canada, ).

Finally, evaluation findings provided some insights on the issue of gender balance in the private space industry. As the involvement of female students is limited in many of the educational fields related to space, it will come as no surprise that this imbalance is carried into the space workforce. As noted in a study on the aerospace and space sectors, "women have low participation in core aerospace occupations involved in design, production, maintenance and repair of aerospace and space products" (Prism Economics and Analysis, , p. 6). A number of factors may contribute to this, including persistent stereotypes such as a perceived conflict between a career in science and family responsibilities (McCreedy & Dierking, , p. 33). Interviews conducted with industry representatives as part of this evaluation were aligned with these trends. It was noted that the space industry remains largely dominated by male employees, and significant barriers remain. As one key informant put it, "the male-female distribution is not at all balanced. For every 20 applicants, I would say that we get one qualified woman. I don't know why that is, but we do not have a balance."

Needs of federal departments and agencies

The evaluation provided limited opportunities to explore the current needs of federal departments and agencies when it comes to recruiting HQP with space education. Considering first the CSA itself, evaluation findings indicate that the agency has not been facing significant challenges when it comes to recruiting the HQP it requires. The decision to largely end internal research within the CSA following the program review is among the factors that have contributed to the limited demand for new HQP to be recruited within the CSA.

A number of other federal departments also use space data and information and, as such, recruit HQP with space-related skills and competencies. The list of these departments and agencies includes (but is not limited to) Environment and Climate Change Canada, Natural Resources Canada, the Department of National Defence, the Department of Fisheries and Oceans Canada, and Agriculture and Agri-Food Canada (Euroconsult, , p. 40). Noting that only some of these federal departments have been directly consulted as part of the evaluation, findings indicate that they do require, from time to time, highly specialized personnel, with graduate or postgraduate credentials, which may not be available in Canada. These individuals must therefore be recruited internationally. In any case, it was noted that there is a limited number of such positions available to start with.

Integrating the space workforce

In concluding this sub-section on the overall context of space-related HQP training and labour force requirements, it is important to acknowledge the widespread perception among those consulted as part of this evaluation that very few employment opportunities currently exist in Canada for integrating the space sector, particularly for those who pursue space science studies at the graduate level. As noted in this sub-section, the extent to which this perception is accurate requires a nuanced response, but the perception itself remains relevant.

Interview findings leave little doubt that, particularly among academics involved in space science, there is a strong view that their students may need to move to other regions of the world if they wish to pursue a space career as a researcher. Their perception is largely based on the following assumptions:

  • The CSA is no longer carrying out internal research and, as such, does not require new space scientists.
  • Academia is largely a closed field with very few openings for new faculty, particularly those wishing to focus on space science.
  • Industry is a cyclical sector that employs few graduate students in space science disciplines, focussing predominantly on engineers.
  • Other federal departments offer limited employment opportunities that are space-focussed.

Not surprisingly, this perception among academics was also echoed by students who were consulted as part of this evaluation. Some interviews were, in fact, conducted with students who now live in other countries to pursue their space interests. It is important to note, however, that these perceptions were not quantified, and the data collected was not meant to be statistically representative. It is, nonetheless, pointing to a perception shared among many key stakeholders that is worth considering as part of this evaluation.

4.1.1.2 Promoting STEM to Canadian youth
Youth interest and achievements in STEM

Over the years, the need to promote STEM among youth has been well-documented and it is a challenge that Canada continues to share with many other countries.

The inherent value of STEM skills is widely accepted. Whether a youth pursues science-related studies or branches out to other fields of study, the fundamentals of STEM can only support his or her ambitions and goals by enhancing his or her abilities in problem solving and reasoning (The Council of Canadian Academies, ). The recent Innovation and Skills Plan released by the federal government reinforced that notion as it emphasized that "the goal of education should be to make every Canadian "innovation ready"—ready to spot opportunities, imagine possibilities, discover new ideas, learn and grow" (Government of Canada, , p. 4). STEM skills go a long way in supporting such innovative behaviours.

Canadian youth tend to score well on exams conducted by the OECD-led Programme for International Student Assessment (PISA). Results from indicate that, out of 65 countries, Japan, Estonia, Finland, and Canada are the four highest-performing OECD countries when it comes to science, mathematics, and reading (OECD, , p. 4).

There are, however, concerning trends. Regardless of the strategic importance of STEM, studies point to a steady decrease over the last 20 to 30 years in the interest of Canadian youth in pursuing STEM-related activities, education, or careers (NSERC, , p. 6). Also, while Canada ranks above average when it comes to the number of students engaged in mathematics and statistics, physical sciences, and life sciences, it ranks below average when it comes to engaging youth in engineering and computing (Department of Finance Canada, , p. 25). Finally, while there has been an increase in women pursuing STEM studies overall, there has been a decrease in recent years in the proportion of women studying mathematics or engineering (Weinrib, , p. 31).

Actively promoting STEM among youth

Recognizing the value of promoting STEM among youth, countries around the world are putting forward a variety of initiatives. For the purpose of this evaluation, it is particularly relevant to point to some of these initiatives that have used space as the vehicle through which STEM is being promoted. As Canadian initiatives are specifically discussed in sub-section 4.1.1.4 of this report, the following paragraphs focus on initiatives implemented in other countries.

US initiatives

In the US, the United States Department of Education has been collaborating with the National Aeronautics and Space Administration (NASA) to provide grants to local schools and community organizations, allowing them to offer STEM activities to students outside of school hours. The 21st Century Community Learning Centers program places a particular focus on NASA's unique space mission to allow students to explore real-life applications of STEM (NASA, 2016a). The primary contribution of NASA has been on training program staff, providing technical assistance, and facilitating the engagement of subject-matter experts. Traditionally, NASA has also provided professional development programs for educators to facilitate their use of NASA curriculum materials (NASA, ), but these activities were being restructured at the time of this report (NASA, 2016b).

Other American-based STEM promotion activities include, for instance:

  • space camps being offered by the US Space and Rocket Center (a museum based in Alabama), which targets youth in grades 4 to 12, and which has engaged 750,000 participants since (U.S. Space & Rocket Center, );
  • the Space Station Explorers Program, offered by the Centre for the Advancement of Science in Space (a government-funded national laboratory managing the US portion of the International Space Station), which targets students from Kindergarten to Grade 12 and offer activities related to robotics, mechanical engineering, biology, data analysis, and computer programming (CASIS, ); and
  • the Go for launch! three-day program offered by the non-profit organization Higher Orbits, which engages students in designing a space experiment and includes a variety of collaborative activities on spaceflights and other related themes (Higher Orbits, ).

Finally, it should be noted that some private space industries also engage in STEM promotion. For instance, United Launch Alliance provides their student interns with the opportunity to volunteer time during their internships to build rockets or payloads, and possibly engage primary and secondary students in the process (United Launch Alliance, ).

Other regions and countries

The following examples illustrate the range of STEM promotion activities offered in other regions of the world:

  • Counting 22 member states that all have their own language and education systems, the European Space Agency (ESA) has opted for an approach that is tailored to each country interested in the STEM initiatives offered through the ESA Resource Office. Interview findings indicate that 13 member states are currently participating in STEM activities offered by the ESA Resource Office, with up to seven more member states in the process of joining by the end of . Working directly with education ministries and curriculum developers, the ESA Resource Office designs tools and activities that target teachers, to facilitate their work in integrating space in STEM education (ESA, n.d.).
  • The CNES offers teacher training through its Université d'été espace éducation, which provides participants with an overview of tools available for education about space, and allow teachers to enhance their knowledge of space (CNES, ).
  • The Australian government has been focussing on Mars to shape its STEM promotion activities, including its Pathways to Space Program, which teaches students about Mars exploration, and offers many hands-on experiences. The stated goal of the program is to inspire students to pursue careers in science and engineering (Dougherty, Oliver, & Fergusson, ).
  • The Japan Aerospace Exploration Agency (JAXA) offers a variety of "bottom-up" activities that are led by participants rather than by the JAXA. For instance, the agency provides assistance to teachers who wish to develop STEM initiatives and invites communities to reach out to the agency if they wish to organize community events promoting STEM. JAXA also offers the Space Mission High School program, which is a five-day training program for high school students where they design and present scientific missions, as well as the Seeds in Space program that allows students to compare the growing processes of flower seeds that have gone to the International Space Station with those that have not (JAXA, ). This latter program is similar to the Tomatosphere program offered in Canada, which is further discussed in sub-section 4.1.1.4 of this report.
  • Finally, a few international initiatives have been designed to promote STEM among youth. For example, the Mission X: Train like an astronaut program involves numerous organizations and space agencies (ESA, NASA, CNES, JAXA) that allow teams of elementary students from different countries (including Canada) to compete through activities that promote healthy eating and exercise through a space lens (Mission X, ). The Zero Robotics program is another such initiative that engages students from various countries who program robots online to solve annual challenges, with the championship conducted by an actual astronaut aboard the International Space Station (Massachusetts Institute of Technology, ).
4.1.1.3 Relevance of Sub-Program activities

Considering the broader context of HQP development and its associated SST knowledge growth, and of the promotion of STEM, this sub-section turns to each component of the Sub-Program and summarizes evaluation findings addressing their specific relevance.

Science and Academic Development (SAD)

As indicated in the description of the Sub-Program (see sub-section 2.2.2 of the report), a number of activities fall, in principle, within the SAD grouping, including FAST, research clusters, industrial chairs, and research infrastructures. In practical terms, however, as SAD funding dropped from roughly $11.8 million in to $4.1 million in , activities became largely centred on FAST. During the evaluation period, the CSA issued one FAST AO in and another one in , but it did not initiate new activities under other SAD sub-components such as the Cluster Program and the Space Science Enhancement Program (SSEP).

The relevance of FAST

In this context, evaluation findings on the relevance of SAD activities predominantly inform the relevance of FAST. These findings leave no doubt as to the critical role that the support allocated through FAST plays in providing hands-on training to HQP university students involved in space science, as well as in supporting the growth of SST knowledge. Both academics and students consulted as part of the evaluation provided illustrations of the added value of projects that had received FAST funding. As one professor put it, "students see an end-to-end mini space program happen by designing, developing, building, testing, flying and analyzing data". This view was echoed by a participating student who indicated that "the research experience you gain from a FAST project is complementary to what you get in university; it is incredibly valuable and, without the CSA funding, it would not be possible".

The parameters of FAST are particularly well-suited to meet the needs of its targeted recipients: having space-like mission projects that systematically involve students and their supervisors, with activities undertaken in a simulated space environment (suborbital or orbital platforms, field sites, or ground-based infrastructures), provides a unique opportunity to build on the theoretical training received in university. Also, the range of research disciplines covered by FAST (satellite engineering, space life science, space astronomy, planetary exploration, Earth system science, and solar-terrestrial science) (Canadian Space Agency, 2015b) is well aligned with the broad definition of what the space sector entails.

Participating academics were particularly appreciative of the fact that FAST projects could involve undergraduate students as well as graduate students. They noted that engaging undergraduate students in these types of projects and activities was probably one of the most efficient strategies to motivate them to pursue graduate studies in a space-related discipline.

The niche that FAST has carved when it comes to funding space-related projects does not, however, respond to all the needs that are described in sub-section 4.1.1.1 of this report. For instance, the program is currently not designed to easily accommodate academics and students who wish to focus predominantly on conceptualizing and designing new measurement instruments, or on the analysis of space data.

Within the niche that FAST does occupy, it is important to note that the funding it provides is unique. Evaluation findings have not identified any other program in Canada that provides these types of opportunities. As further discussed in sub-section 4.1.1.4 of this report, other programs offered by the CSA and other granting institutions provide important support for those engaged in space studies, but none of these duplicate what is offered through FAST.

Other SAD components

While applicable evaluation findings are more limited, they do indicate that other components that were previously funded under the SAD umbrella responded to needs that remain relevant when it comes to HQP development and SST knowledge growth. The support that the CSA provided through the Cluster Program and the SSEP offers particularly helpful illustrations in that regard.

In addition to providing an avenue for researchers from different institutions to collaborate, the Cluster Program allowed participants to focus more predominantly on advancing scientific knowledge related to space. As such, AOs issued under the Cluster Program did not require end-to-end projects involving demonstration activities. Rather, recipients could design activities involving both academics and students in conducting scientific research in their field of study. For the period covered by this evaluation, one Cluster AO was issued in , which led to 10 grants being provided.

Along the same lines, the SSEP provided funding for activities that are complementary to those covered under FAST. Instead of focussing on end-to-end projects, SSEP provided funding to conduct initial concept studies on space science instruments, space data analysis, and space science-related academic activities (Canadian Space Agency, ). Activities under the SSEP occurred prior to the period covered by the evaluation, as the last AO issued for this program dates back to . However, at the time of the evaluation, there were over 30 SSEP grants that were still active, the majority of which were closed in .

Turning to the Canadian Analog Research Network (CARN), this former component of SAD focussed specifically on providing small grants (approximately $50,000) allowing students and academics to travel to and work from analog sites located in Canada (e.g., Arctic Region) or in other countries. The CSA issued the last CARN AO in and, during the early part of the period covered by the evaluation, only a handful of grants were still active. Interview findings indicate that this type of funding is not readily available from other granting institutions, and is particularly well-suited to the goals pursued under SAD to train HQP and advance SST knowledge.

Finally, the CSA has been providing funding through SAD to support industrial chairs in space-related fields. This type of funding normally involves, in addition to the CSA, the National Sciences and Engineering Research Council of Canada (NSERC), universities, and private industries. As such, this funding does not result from formal AOs, but is instead provided as unsolicited grants, based on opportunities that arise. An illustration of such funding is the NSERC Multisectorial Industrial Research Chair in Coatings and Surface Engineering (MIC-CSE), launched in (Polytechnique Montréal, ). With a total budget of $5.4 million over five years, this industrial research chair involves the Polytechnique Montréal, NSERC, the CSA, and Hydro Québec, along with five private industries. The CSA has contributed $150,000 to this project. During the period covered by the evaluation, the CSA provided four funding allocations to support industrial research chairs, with total amounts varying between $150,000 and $500,000. Interview findings indicate that having the three pillars of the space sector (government, academia, and industry) collaborating together can be highly valuable. As one key informant noted, however, there is a limited number of space companies in Canada that can engage in long-term research projects, so the scope of this type of project is bound to be fairly limited.

This brief review of the various components included under SAD indicates that, while FAST remains highly relevant in its current format, other types of support can also make significant and complementary contributions to training HQP and enhancing SST knowledge in the various fields falling within the scope of the space sector. From a strict programming perspective, this does not necessarily mean that new programs must be created or former programs reinstated. In fact, FAST could potentially accommodate some of these needs through modified eligibility criteria, if such was the direction that the CSA opted to pursue.

STRATOS

Analog sites, planetary rover prototypes, parabolic flights, stratospheric balloons, sounding rockets, or nano/microsatellites are all demonstration opportunities that support hands-on training in key space-related fields of study. In some instances, such as the FAST AO that the CSA issued in , conducting a planned field campaign at an analog site using a specific ground-based infrastructure or participating in a suborbital/orbital flight was a mandatory condition for obtaining funding, which illustrates the value placed on such activities.

During the period covered by this evaluation, one of these demonstration activities – the CSA stratospheric balloon program STRATOS – was specifically included under the Sub-Program. Evaluation findings confirm that this program is highly relevant and provides critical support to a range of learning opportunities for HQP. Interview findings emphasized how particularly well-suited experiments using stratospheric balloons are for students engaged in graduate studies. It directly supports the concept of end-to-end space projects that can be completed in a near-space environment within two to three years, and upon which master's or doctoral theses can be written. It also represents one of the most practical phases of end-to-end projects, as students engage with representatives of the CSA, the CNES, and, in some cases, private-sector partners, to launch and manage the flight and its associated data collection.

As illustrated in the case study report on STRATOS, these stratospheric balloon experiments also advance SST knowledge. In that specific case, the ALI instrument developed and tested through STRATOS became the first suborbital demonstration of an imaging aperture-acousto tunable filter for atmospheric remote sensing. It also contributed to increasing the applicable technology readiness levels (TRL).

In addition to being fairly flexible, projects based on stratospheric balloon flights are less expensive than many other demonstration alternatives. This was noted throughout interviews conducted as part of this evaluation, as well as by the review of the space sector led by David Emerson in (Aerospace Review, , p. 42).

The relevance of STRATOS must also be examined through the partnership with the CNES upon which the program rests. From both the Canadian and French perspectives, the partnership is highly appreciated, as it allows for an exchange of expertise and knowledge in addition to providing the required infrastructure to conduct stratospheric flights in Canada. As noted by the CNES representatives consulted, Canada offers more than just a location from which stratospheric balloons may be launched; it also provides a fully functioning operation centre and collaboration with CSA representatives. There is a strong desire on the part of the CNES to maintain and potentially expand the program to offer new opportunities for stratospheric experiments.

It should be emphasized that there are no other stratospheric balloon programs offered in Canada. As noted by one key informant, "before STRATOS, there was no way for Canadians to be involved in stratospheric ballooning, unless you had established outstanding collaborations with NASA". Evaluation findings do confirm that some Canadian space scientists are involved in stratospheric experiments involving other space agencies, but these are fairly exceptional, and do not duplicate what STRATOS is offering.

Finally, interview findings indicate that widening the range of experiments that can be tested through STRATOS (in terms of payload weight and duration of flights) would further open the range of research projects that could be conducted through this demonstration avenue.

Engineering Development Support (ED1)

Delineating the range of activities that falls within the ED1 component of the Sub-Program is a rather challenging task, which in turn limits one's ability to provide a comprehensive assessment of its relevance or performance. As noted in the description of ED1 (see sub-section 2.2.2.2), the component's overall goal is to allow a group of engineering experts within the CSA to provide direct support to various internal and external stakeholders involved in space-related projects, activities, or studies. This group also supports the involvement of the CSA in a number of international fora such as the Inter-Agency Space Debris Coordination Committee, the European Cooperation for Space Standardization, or the International Astronautical Federation. Before the budget reduction implemented through the program review, the ED1 group also supported internal research projects involving post-doctoral fellows hired under the NSERC Visiting Fellowships in Canadian Government Laboratories Program.

If the range of activities undertaken through the ED1 component remains fluid, the fact remains that the technical expertise found within this group is a critical asset of the CSA, which serves the agency at many levels, including in pursuing the goals of the Sub-Program when it comes to HQP development and SST knowledge growth. By (among other things) acting as external examiners or co-supervisors of theses, supporting engineering capstone projects, assisting STRATOS campaigns or other forms of demonstration activities, and assisting in the review of proposals submitted for funding under different AOs, the ED1 group supports the level of technical quality that space-related projects are expected to demonstrate.

Junior Engineers Training Program

The relevance of the Junior Engineers Training Program can be assessed from two perspectives. For newly graduated engineers interested in a career in space, the program constitutes an extraordinary opportunity. As made abundantly clear by former program participants now working at the CSA, this program truly exceeded their initial expectations. They emphasized that the CSA is perceived as being a particularly difficult organization to join, having very few job openings. Having access to the Junior Engineers Training Program was therefore seen as an exceptional opportunity.

From the perspective of the CSA itself, the program does support its efforts to renew its internal engineering capacity. Program participants are exposed to a variety of engineering functions within the CSA and, based on each participant's educational credentials and experience, they can be placed within the appropriate branch of the agency. What evaluation findings also indicate, however, is that there are limitations to what program participants can offer. Just like any other organizations, CSA employees with significant experience will eventually retire, and the agency needs strategies to ensure that the level of technical expertise required by its missions and activities can be secured. Participants in the Junior Engineers Training Program can contribute to some extent, but not completely, in filling this gap. As a result, the program remains relevant for the CSA, but it constitutes one component of a larger solution that includes other components to ensure the proper renewal of the agency's engineering expertise.

Awareness and Learning and Space Learning

The need to promote STEM among Canadian youth is well-established (see sub-section 4.1.1.2). At the time of this evaluation, the CSA was undertaking a number of initiatives targeting Canadian youth, but the Sub-Program itself was no longer actively engaged in this area. The Sub-Program's contribution was more focussed on supporting specific activities for university students (such as attending space-related conferences or through projects funded by the SAD component).Consequently, the question becomes one of determining the extent to which it is appropriate for the CSA to undertake new and additional activities targeting youth.

Evaluation findings indicate that, beyond the various communication, outreach, and promotional activities currently being delivered, including those that target Canadian youth, the CSA needs a more concerted and coordinated effort to promote STEM through space, and space through STEM. As demonstrated by other space agencies, and noted during interviews, this means providing both the tools and the required support to use these tools. As fascinating as space can be, it remains a complex field that requires professional pedagogical approaches. At the time of this evaluation, the CSA was further structuring its awareness and learning activities through a STEM plan that is expected to cover activities targeting both Canadian youth and students, and through revisions to the component of its Class G&C Program relating to Awareness and Learning.

The bottom-up approach adopted by other countries such as Japan and, more generally, the ESA (see sub-section 4.1.1.2 for details), may appear particularly relevant in the Canadian context. By providing various options to engage Canadian youth and allowing initiatives to emerge from those engaged with them, the CSA would be able to provide the type of assistance that is consistent with its fundamental mandate of promoting space and advancing the knowledge of space (Statutes of Canada, , sec. 4). While other organizations make important contributions when it comes to promoting space (as discussed in the next sub-section), the CSA stands apart in terms of credibility, knowledge, and appeal. The CSA is the organization that Canadian astronauts report to, and it engages in a variety of space missions. Just as with other space agencies, and noted during interviews, this important asset is most likely to engage Canadian youth.

4.1.1.4 Other key contributions

As frequently noted throughout the report, the development of HQP, the advancement of SST knowledge, and the promotion of STEM and space among Canadian youth are all endeavours that engage stakeholders beyond the Sub-Program and the CSA itself. The purpose of this report is not to assess the efficiency of these activities undertaken by other groups within the CSA or external stakeholders, but rather to acknowledge their expected contribution, recognizing that they ultimately complement the role that the Sub-Program plays in that regard. This sub-section does not provide a comprehensive listing of these initiatives, but rather highlights those that emerged more systematically from the various lines of evidence.

Development of HQP and SST knowledge growth

Other activities within the CSA

It would be reasonable to argue that all activities undertaken through the CSA's three main program branches (Space Utilization, Space Exploration, and Space Science and Technology) contribute, at some level, to the development of HQP and to the growth of SST knowledge. In many cases, the development of HQP is not a stated or primary goal, but it does make a contribution nonetheless. Using the number of projects allocated mostly through G&Cs or contracts by the CSA to academic researchers as one indicator, administrative data indicates that, over a four-year period ( to ), all three branches have supported fairly equivalent numbers of such projects, as illustrated in Figure 4.

Figure 4: Number of projects allocated to academic researchers between 2012-13 and 2015-16, per CSA program branch
Space Utilization Space Science &
Technology
Space Exploration
Number of projects 196 187 186

(Source: Unitas data)

Within the SST Branch itself (which manages the Sub-Program), the Space Technology Development Program (STDP) provides a telling example of such complementary contribution (Canadian Space Agency, 2016h). The STDP provides financial support to entities, mostly private-sector industries, to develop specific space technologies. The two primary goals of the program are to develop mission-enabling and generic technologies to support future needs of the Canadian Space Program, and to support industrial capability-building. In doing so, the program necessarily contributes to SST and the development of a highly skilled workforce.

Through the various activities and projects they undertake, the Space Exploration and Space Utilization branches also provide multiple opportunities to engage academics, students, and industries.

The use of Synthetic Aperture Radar (SAR) data provides one illustration of the contribution that Space Utilization activities are making to train HQP and a space-related workforce. The CSA manages a number of programs that facilitate the use of SAR data produced by the RADARSAT-2 satellite. The list of these programs includes:

  • the Government Related Initiatives Program (GRIP), which supports the use of SAR data by other federal departments and agencies;
  • the Earth Observation Application Development Program (EOADP), which supports the use of SAR data by the private space sector; and
  • the Science and Operational Applications Research (SOAR) program, which supports the use of SAR imagery by universities. The SOAR program is particularly relevant for the purpose of this evaluation, and it is worth noting that during the period covered by the evaluation, the SOAR program funded six research projects that involved four universities, in addition to providing access to SAR imagery to more than 294 research projects in Canada and in other regions of the world. A survey conducted in by the SOAR program team revealed that more than 300 postgraduate students in Canadian universities were involved in SAR (Canadian Space Agency, 2017c).

As for Space Exploration, the case study on the Mars Sample Return Simulations, conducted as part of this evaluation, provides another illustration of HQP development. Through an AO issued in , the Space Exploration branch provided funding to Western and McGill universities to undertake space exploration Science Definition Studies (Canadian Space Agency, 2016i). These activities were integrated in the larger simulation initiative led by Western University that involved a total of 32 students, mostly at the graduate level, along with 12 researchers and professors from five Canadian universities, as well as representatives from private space industries and other space agencies (NASA and the United Kingdom Space Agency). While the primary goal of Science Definition Studies is to demonstrate technologies and advance science, they systematically support HQP development.

In addition, the CSA has, over the years, integrated students through co-op programs, the Federal Student Work Experience Program (FSWEP), the Research Affiliate Program (RAP), or other similar initiatives. While these students – typically undergraduate and graduate students – have been assigned to various groups within the CSA, interview findings indicate that both the Space Exploration and Space Utilization branches have employed a significant portion of them.

As Figure 5 indicates, the number of students who have been involved with the CSA dropped significantly following the program review. From having close to 200 students working in various programs in , the number decreased to 44 by .

Figure 5 : Distribution of students within the CSA, per categories and fiscal years
- - - - -
Coop 133 82 39 34 36
FSWEP 28 23 9 6 6
Other 34 19 2 2 2

(Source: CSA demographic profile data)

From time to time, the CSA has also provided funding assistance to allow students to participate in space-related conferences. For instance, in , the CSA allocated funding through its Student Participation Initiative to assist students wishing to attend the International Astronautical Congress of the International Astronautical Federation (Canadian Space Agency, 2016j). This funding was open to all university students (bachelor's, master's, doctoral, and post-doctoral).

Activities funded by NSERC

NSERC programs are expected to make a significant contribution to the development of HQP in science and engineering fields. While the range of research fields covered by NSERC programming extends beyond space, it does include it, and a number of space-related projects have been funded by both NSERC and the CSA.

As noted in the preceding sub-section, the CSA has been engaged in industrial chairs that have received NSERC funding. Interview findings also indicate that other NSERC funding, such as the Discovery Grants Program and the Engage Grants Program, have been used by academics in space science fields to involve their students.

It is, however, the NSERC Collaborative Research and Training Experience (CREATE) program that has been mentioned most often during interviews conducted as part of this evaluation. Allocated yearly, the CREATE program provides up to $1.65 million over a six-year period to each of its recipients (Natural Sciences and Engineering Research Council of Canada, 2016b). One key characteristic of the program is that at least 80% of the grant provided must be used for trainees' stipends. While this is seen as providing a significant opportunity to enroll students, interview findings indicate that it also limits the range of activities that this program can support. As noted by academics who were consulted, this is where CSA funding such as FAST or STRATOS plays a critical role in allowing other types of costs or activities to be covered in order to provide more meaningful end-to-end experiences.

The case study on the Mars Sample Return Simulations provides another illustration of how NSERC funding, combined with other CSA support, can make a significant contribution to space-related HQP development and knowledge growth. As part of this specific project, in Western University received the maximum amount of $1.65 million over a six-year period to implement the NSERC CREATE Technologies and Techniques for Earth and Space Exploration program. In addition to the funding it provided through a Space Exploration Science Definition Studies AO, the CSA supported this sample return simulation project by providing access to CSA rovers, test sites, operation facilities, and subject-matter expertise.

The inherent need to combine multiple sources of funding to carry out hands-on research projects, including those related to space studies, emphasizes the critical importance for funding agencies to collaborate closely. Interview findings indicate that the relationship between NSERC and the CSA is particularly critical in that regard, and there was a strong desire expressed to see even closer links between the two organizations to ensure that the planning and the implementation of their respective programs can be coordinated to maximize its contribution to HQP development and SST knowledge growth.

Other relevant activities in Canada

Many other activities taking place in Canada have roles that are complementary to what the Sub-Program is undertaking. In some cases, the CSA offers some support, whereas in other cases these activities are carried out independently. The following are those that were mentioned more often by individuals consulted as part of this evaluation:

  • The Polar Environment Atmospheric Research Laboratory (PEARL): Operated by the Canadian Network for the Detection of Atmospheric Change (CANDAC), PEARL is a research laboratory located on Ellesmere Island, 15 km from Eureka and about 1,100 km from the North Pole (Canadian Network for the Detection of Atmospheric Change, ). Its primary purpose is to study the changing atmosphere over Canada. To this end, it hosts a variety of instrumentation that measure atmospheric properties from the ground to around 100 km. PEARL's funding partners include the CSA, NSERC, Environment and Climate Change Canada, the Canadian Foundation for Innovation, the provincial governments of Ontario and Nova Scotia, and a number of research organizations. A total of nine universities participate in the research activities held at PEARL, including both academics and students who have published or delivered multiple publications and presentations over the years.
  • CARIC/CRIAQ: The Consortium for Aerospace Research and Innovation in Canada (CARIC), and its counterpart the Consortium for Research and Innovation in Aerospace in Québec (CRIAQ), focus on bridging private industries and academics involved in aerospace. They both provide research funding for applied research that aims to advance the TRL of emerging technologies.
  • Mitacs: Established as a national not-for-profit organization, Mitacs designs and delivers a number of research and training programs. Typically involving matched funding between government and industry sources, these programs provide students and post-doctoral fellows with opportunities such as internships (Accelerate), research management training (Elevate), international fellowships (Globalink), workshops (Step), and policy-oriented fellowships (Canadian Science Policy Fellowship) (Mitacs, ). Interview findings confirm that academics and students involved in space science have participated in some of these programs.

Other relevant activities outside of Canada

Finally, a number of activities held outside of Canada engage Canadian HQPs. Among those mentioned by individuals consulted as part of this evaluation is the Canada-Norway Student Sounding Rocket (CaNoRock) exchange program, which is a partnership between the universities of Alberta, Calgary, and Saskatchewan, the University of Oslo, the University of Tromsø, the Andøya Space Center, and the Norwegian Center for Space Related Education in Norway (Andøya Space Center, ). As noted by one professor consulted, this exchange program provides a rare opportunity for undergraduate participants to engage with students from other countries in hands-on space-related activities. The CSA has provided funding in support of CaNoRock.

The International Space University (ISU) is offering a number of academic and professional development activities in its primary location in Strasbourg, as well as in other locations around the world (International Space University, ). The Master of Space Studies and the two-month Space Studies Program have been favorably mentioned during interviews conducted as part of this evaluation. While the CSA provided funding in the past to assist students who enrolled with the ISU, this funding was eliminated as part of the program review.

Finally, evaluation findings indicate that one learning opportunity that may not be sufficiently known among Canadian students is the Training and Learning Programme offered by the ESA (European Space Agency, ). Since Canada is an Associate Member of the ESA, Canadian university students have full access to this hands-on program that covers space research, facilities, and applications for science and engineering. The challenge, particularly for Canadian students, is to secure the required funding to cover related expenditures such as travelling costs.

STEM promotion

As previously noted, at the time of this evaluation, the Sub-Program was no longer involved in promoting STEM and space among Canadian youth. While sub-section 4.1.1.2 of this report describes initiatives undertaken in other countries to promote STEM, the following paragraphs summarize evaluation findings related to initiatives held in Canada, within and beyond the CSA, that are relevant for the purposes of the evaluation of the Sub-Program.

Activities within the CSA

It is predominantly through the activities of its Communications and Public Affairs Directorate that the CSA is reaching out to the Canadian public, including youth (mostly from elementary and secondary grades), and engaging them in activities that highlight the various dimensions of space. Without aiming to be exhaustive, the following list provides a good illustration of the nature and scope of these outreach activities conducted during the period covered by this evaluation:

  • Since , in collaboration with the Canada Aviation and Space Museum, the CSA has developed the exhibits "Living in Space" and "Canadarm," which allow visitors to gain a greater appreciation of life on board the International Space Station. According to CSA's administrative data, over 560,000 visitors have seen these exhibits to date.
  • Launched in , the five-month mission to the International Space Station (ISS), involving Canadian astronaut Chris Hadfield, provided a unique opportunity to engage all Canadians, but especially Canadian youth. Among the many activities organized around this mission were 24 live events with students from all regions of the country. One of these events was a live concert from space with Chris Hadfield, which attracted close to one million participants across Canada. Also, a total of 88 videos addressing science and weightlessness in space were produced during this mission, and have been viewed more than 40,000 times.
  • In , the CSA produced its five "Canada from Space" giant maps for educators, which were produced in collaboration with the Royal Canadian Geographical Society and the Canada Aviation and Space Museum. These maps allow Canadian students to get a better understanding of the role played by Earth observation satellites. To date, 51 elementary and secondary schools have participated in this initiative.
  • On an ongoing basis, Canada's two astronauts participate in many outreach events with Canadian students. For instance, administrative data indicate that these astronauts offered close to 125 presentations to over 22,000 students in , in addition to participating in five tours across the country.
  • The CSA maintains a strong social media presence. In , administrative data indicate that the agency's YouTube channel counted over 220,000 followers, its Twitter account counted close to 215,000 followers, and its Facebook page counted close to 92,000 followers.

It should finally be noted that the education modules that were created through the former CSA Space Learning Program continue to be used by teachers across elementary and secondary schools in Canada.

Other Canadian-based activities

In addition to the CSA, a number of organizations are actively engaged in promoting STEM and space among Canadian youth. Again, some of these activities are undertaken with the direct collaboration or support of the CSA, while others are carried out independently. In terms of funding, it is worth noting that NSERC provides some financial assistance for STEM promotion through its PromoScience Program (Natural Sciences and Engineering Research Council of Canada, 2016a).

One of these outreach organizations is Let's Talk Science, a Canadian charitable organization founded in that delivers a range of STEM-related programs and activities. Of particular interest for the purpose of this evaluation is the Tomatosphere program. First established by the CSA and transferred to Let's Talk Science following the implementation of the program review, Tomatosphere involves students from Kindergarten to Grade 12. Each year, participating students explore the effects of space on food using the germination of tomato seeds, including some that have travelled into space. Using both treated (space exposure) and untreated seeds, the students conduct scientific experiments and inquiries. In , over 17,700 classrooms participated in the program.

Another charitable organization, Actua, also engages Canadian youth in STEM programming. Using a network of university and college members, the organization reaches an estimated 225,000 youth from all regions of the country, in addition to employing approximately 1,000 undergraduate students (Actua, ). These activities include camps, workshops, and clubs. The organization also implements activities focussing on target audiences, such as the InSTEM program offered to Indigenous youth, and the National Girls Program for female students.

Some Canadian universities also implement STEM outreach programming. For instance, the University of Toronto's Faculty of Applied Science and Engineering offers the Da Vinci Engineering Enrichment Program, which takes the form of summer courses for high school students, and courses taught over three Saturdays for grade three to grade eight students (Faculty of Applied Science & Engineering, University of Toronto, ). In , the summer program offered two courses that were specifically about space (It is Rocket Science and Introduction to Spacecraft and Orbital Mechanics). Western University also offers outreach activities, including sessions for youth on astronomy, planetary science, and space (Centre for Planetary Science and Exploration, ). It also offers space camps to youth aged nine to 14.

4.1.2 Alignment with CSA and government priorities

Promoting STEM, advancing SST knowledge, and developing HQP are priorities of the federal government and of the CSA. These objectives are directly aligned with the emphasis placed on furthering Canada's plan related to innovation.

Federal priorities

At the time of this evaluation, the goals of developing HQP, advancing space science knowledge, and promoting STEM among Canadian youth were all directly aligned with the priorities of the federal government. As noted during interviews, this constitutes a significant shift when considering the entire period covered by the evaluation. The cost-reduction measures implemented as part of the program review greatly reduced the ability of the CSA to engage in achieving all three outcomes, but this policy landscape has clearly evolved.

In , in the mandate letter he provided to the Minister of Science, the Prime Minister set the tone: "We are a government that believes in science – and a government that believes that good scientific knowledge should inform decision-making" (Office of the Prime Minister, 2015b). The letter emphasized that one of the federal objectives when it comes to science is finding "an appropriate balance between fundamental research to support new discoveries and the commercialisation of ideas."

The recognition that innovation is a fundamental pillar of economic growth was also echoed in the mandate letter provided to the Minister of Innovation, Science and Economic Development, whose portfolio includes the CSA. Among other things, it tasks the Minister to work with a range of stakeholders, including post-secondary institutions, "to improve the quality and impact of our programs that support innovation, scientific research and entrepreneurship" (Office of the Prime Minister, 2015a).

Shortly after, in , the federal government released its Innovation and Skills Plan, which rests on six pillars, including the development of a highly skilled workforce (Entrepreneurial and Creative Society), the enhancement of Canadian science capabilities (Global Science Excellence), and the establishment of clusters involving businesses, research institutions, and government (World-Leading Clusters and Partnerships) (Government of Canada, , pp. 4–5). In support of this, the federal government also unveiled Canada's new Global Skills Strategy, which aims to facilitate "faster access to top global talent for companies doing business in Canada that are committing to bring new skills to Canada and create more Canadian jobs" (Innovation, Science and Economic Development Canada, 2017c).

In the fall of , as part of the Canadian Aerospace Summit, the Minister of Innovation, Science and Economic Development emphasized the significant contribution that the aerospace and space sectors are making to innovation, and confirmed that the Space Advisory Board would be revitalized and asked to support the development of long-term priorities for the space sector in Canada, including a new space strategy. Among other things, the new space strategy is expected to support "talent, research and entrepreneurship within the industry" (Innovation, Science and Economic Development Canada, ).

More recently, in , the federal government launched its Choose Science initiative that supports the promotion of STEM among Canadian youth, with a particular emphasis on young women (Innovation, Science and Economic Development Canada, 2017a). The initiative focusses on social media outreach, along with provision of education material for parents, teachers, and mentors (Innovation, Science and Economic Development Canada, 2017b).

These various commitments were further advanced when the federal government tabled its budget in . Recognizing that "big challenges demand new and innovative solutions," the budget unveiled measures to strengthen science in the federal government. In addition to establishing a Chief Science Advisor, the federal government will be developing a new federal science infrastructure strategy, which will include a review of federal investments "in federal science infrastructure, including federal laboratories and testing facilities" (Government of Canada, , p. 88).

The federal budget also included new investments related to space that will contribute to advancing SST and developing HQP:

Canada has a long and proud history as a space-faring nation. As our international partners prepare to chart new missions, Budget proposes investments that will underscore Canada's commitment to innovation and leadership in space. Budget proposes to provide $80.9 million on a cash basis over five years, starting in , for new projects through the Canadian Space Agency that will demonstrate and utilize Canadian innovations in space, including in the field of quantum technology as well as for Mars surface observation. The latter project will enable Canada to join the National Aeronautics and Space Administration's (NASA's) next Mars Orbiter Mission. (Government of Canada, , p. 90)

On the promotion of STEM among Canadian youth, the federal budget unveiled two new measures: an increase in the budget allocated to NSERC's PromoScience Program to expand the reach of activities delivered by its beneficiary organizations, including activities targeting underrepresented groups in STEM. The budget also included a broadening of the Prime Minister's Awards for Teaching Excellence to include 17 new STEM-themed awards (Government of Canada, , p. 74).

CSA priorities

As it relates more specifically to CSA's priorities, the period covered by this evaluation began with the review of the aerospace and space sectors announced in the federal budget (Government of Canada, , p. 86). What came to be known as the Emerson report included a number of recommendations, including one on the need for the CSA to establish a clearer vision of Canada's goal as it relates to space: "in a sector whose undertakings are, by definition, long-term, expensive, and complex, it is especially important to have concrete goals, predictable funding, and orderly implementation" (Aerospace Review, , p. 26).

Partly in response to this review, in the CSA unveiled its Space Policy Framework. The vision articulated in this framework rests on five core principles, including the need to enhance key capabilities (e.g., telecommunications, remote sensing, robotics), and to inspire Canadians to pursue careers in space (Canadian Space Agency, 2014b, p. 10). The framework includes four avenues of strategic actions, including a commitment to work with industry and the space research community to support further opportunities in research and development, and innovation (Canadian Space Agency, 2014b, p. 11).

These findings confirm that the Sub-Program is in a position to support federal priorities related to innovation, including the space priorities that the CSA has identified.

4.1.3 Distribution of roles and responsibilities

Activities undertaken through the Sub-Program fall within the CSA's legislative mandate. There are, however, a number of other internal and external actors that are actively engaged in pursuing the Sub-Program's objectives, which highlights the complementary nature of these activities.

Evaluation findings described in the preceding sub-sections provide ample evidence as to the range of stakeholders that are actively engaged in developing HQP, advancing space knowledge, and promoting STEM among Canadian youth. Reaching the most efficient equilibrium in the distribution of roles and responsibilities among all these organizations is bound to be an ongoing process that reflects evolving mandates, activities, and opportunities.

Exploring more specifically the role that the CSA is expected to play, it is helpful to recall the fundamental mandate that Parliament has assigned to the CSA, as reflected in the Canadian Space Agency Act: "The objects of the Agency are to promote the peaceful use and development of space, to advance the knowledge of space through science and to ensure that space science and technology provide social and economic benefits for Canadians" (Statutes of Canada, , sec. 4).

As already noted in sub-section 4.1.1.1, the expected results of the Sub-Program are directly linked to the notions of advancing "the knowledge of space through science" and ensuring that space science provides "social and economic benefits to Canadians." In turn, given the complexity of space science, it logically requires measures to build, maintain, and enhance the pool of HQP involved in space, from the moment a child engages with his or her broader environment, up to his or her enrolment in highly specialized graduate studies in one of the many fields of study associated with space.

On the specific issue of promoting STEM among Canadian youth, there is already a fairly broad and active network of organizations that are engaging youth in STEM and playing complementary roles. Ministries of education, school boards, and schools expose Canadian youth to the nature and applications of STEM fields. Other organizations, including a number of non-profit organizations that have a particular focus on space, are called upon to expand these STEM opportunities. As for the CSA, it stands as the primary centre of space expertise in Canada. In this context, the ongoing contribution of the CSA must be targeted and complementary, in addition to falling within its legislated mandate. In doing so, the CSA can capitalize on the fact that its assets have no equivalent, and that the organization holds, or has access to, a level of space expertise that is unmatched in Canada. Building on the range of communication activities that it already undertakes, the CSA can collaborate to more systematically promote STEM through space, and space through STEM. The tailored and targeted approach used by other agencies in pursuing this goal (as noted in sub-section 4.1.1.2 of this report) is particularly appealing in the Canadian context.

4.2 Design and delivery of the Sub-Program

This sub-section of the report summarizes evaluation findings on the design and delivery of the Sub-Program. It covers the mechanisms and processes in place to deliver the Sub-Program components, and the current strategy to measure their ongoing performance. This sub-section also covers the gender-based requirements applicable to the Sub-Program. Since the activities of the Sub-Program on STEM promotion directed at Canadian youth were no longer active at the time of the evaluation, they are not covered by this sub-section.

4.2.1 Design and delivery structure

The CSA has established sound processes and structures to deliver the set of activities covered by the Sub-Program. The evaluation identified opportunities for improvement related to SAD's current delivery capacity and the ED1's activity planning process.

SAD component

Evaluation findings on the delivery of SAD activities essentially focus on the AOs issued under FAST, as key informants were not in a position to offer insights on previous AOs that were issued under other components of SAD in the early part of the period covered by the evaluation.

Overall, funding recipients perceived the management of FAST AOs positively. They systematically emphasized the professionalism of CSA representatives and their willingness to offer assistance as needed. Interview findings also indicate that funding recipients were generally familiar with the range of activities covered by FAST AOs and were in a position to prepare their submission accordingly.

Interview findings also pointed to some challenges related to the FAST AO process. On several occasions, internal and external key informants encouraged the CSA to predominantly use AOs to provide funding support (as opposed to unsolicited proposals), and to issue its FAST AOs on a more predictable basis. This finding is directly in line with the findings of the evaluation of the Class G&C Program discussed in sub-section 2.3. The other main challenge raised was around the burden associated with the proposal and reporting requirements. Some key informants were of the opinion that the amount of information required in proposals was too onerous, particularly when considering the level of funding provided. There was also a general perception among funding recipients interviewed that the reporting requirements were particularly burdensome, and they were unsure as to how the CSA was using this information.

Somewhat linked to the perceived unpredictability of AOs, interview findings indicate that coordinating activities between FAST and STRATOS, or funding between FAST and other granting organizations, remains a challenge for many recipients. A number of key informants suggested the CSA work more closely with other granting organizations such as NSERC or the Canadian Foundation for Innovation, and other federal departments as applicable, to enhance the complementarity of their respective funding programs and, possibly, their funding calls and reporting requirements.

On the timeliness of the allocation process, interview findings indicate that one area of concern is the delay experienced between the time the results of an AO are announced and the establishment of agreements, and the associated flow of financial resources. Evaluation findings indicate that several weeks may be required to complete this final step, which restricts the ability to promptly undertake the funded projects.

As for the CSA's internal capacity to deliver the SAD component, evaluation findings indicate that the SAD group is already struggling to monitor all current projects, offer the required support to funding recipients, manage unsolicited proposals, and appropriately plan for future AOs. In this context, it would not appear reasonable to expect that the CSA could provide, if such was its desire, more predictable AOs as recommended by many of the participating stakeholders.

STRATOS

The design and delivery structure of STRATOS was described positively throughout the interviews conducted as part of this evaluation. STRATOS was regularly described as a being an exemplary program when it comes to supporting HQP development and SST knowledge growth.

Among the factors contributing to this success is the partnership between the CSA and the CNES. Representatives from both agencies highly value this working relationship and the complementarity of experience and knowledge that derives from it. In fact, representatives of the CNES do not only wish to see the partnership continue, but they would welcome opportunities to expand it. An example provided was the possibility of having joint projects between France and Canada, as opposed to only sharing available space on stratospheric balloons.

Both professors and students who had participated in a STRATOS campaign, and who were consulted as part of this evaluation, emphasized the professionalism and commitment of CSA representatives in providing the required support to ensure that their payload could be placed on a stratospheric balloon and that their mission could be successfully carried out. Even when unexpected difficulties arose (such as delays resulting from adverse weather conditions), the STRATOS team worked closely with participating teams to manage the impact of these delays on their experiments.

When asked if STRATOS could better meet their needs, some key informants noted that the current parameters of the program in terms of the weight of eligible payloads and the duration of flights limit the range of experiments that can be conducted. As a result, broadening the capacity of these stratospheric balloons would, in turn, open up new opportunities that, at the time of the evaluation, could only be sought in other jurisdictions, such as NASA.

Internal activities

Both the ED1 and the Junior Engineers Training Program are managed internally. As such, the evaluation gathered some findings related to their delivery, but the scope of this information remains limited.

As previously noted, the delivery of ED1 activities varies in accordance with internal requirements and external opportunities. While some tracking of time has been done to better understand how ED1 activities were evolving, it only provided a partial picture of how these activities were being planned and managed. Evaluation findings indicate that this working framework is in the process of changing to become more systematically structured. As such, the Engineering Development Directorate is implementing new processes that will ensure that all activities undertaken as part of the ED1 component of the Sub-Program are accounted for and linked to an established priority of the CSA.

As for the Junior Engineers Training Program, only two rounds of recruitment had been completed at the time of the evaluation, the first one involving eight participants and the second one involving four participants. In this context, the program is still evolving. A key difference between the two rounds of recruiting is the fact that the first one advertised for indeterminate positions, while the second one advertised for determinate positions.Footnote 10 As noted during interviews, this could explain, at least in part, the fact that over 1,000 applications were received during the first round, while 400 applications were received during the second round. Participants in the program who were consulted as part of this evaluation did emphasize the quality of the recruitment process and of the ongoing support provided throughout their various assignments. As noted in the case study report, one suggestion for improving the delivery of the program would be to complete the selection process in a timelier manner, to allow candidates to pursue other avenues in case they are not selected.

4.2.2 Performance measurement strategy

The CSA has established a performance measurement strategy for the Sub-Program, which has yet to be fully implemented.

During the period covered by the evaluation, there was no performance measurement strategy in place in relation to the Sub-Program. However, the CSA developed a strategy that became operational as of (Canadian Space Agency, 2016e). This tool includes all key components required by such a strategy, including a list of indicators that will allow for the activities and some of the results achieved under the Sub-Program to be documented in a systematic and ongoing manner.

As noted in the Performance Measurement Strategy, this evaluation of the Sub-Program includes a series of findings and insights that could provide assistance in refining the strategy, clarifying what can be expected to be measured as part of the strategy, what is expected to be covered by a formal evaluation process, and how the two can act in a complementary manner.

Also noted in the Performance Measurement Strategy is the fact that resources required to fully implement the strategy have yet to be identified. Along with addressing this issue, it would be beneficial for the CSA to also confirm how performance data will be collected, analyzed, stored, and shared, both as part of the ongoing management of the Sub-Program and also during the next evaluation of the Sub-Program. For instance, creating annual performance reports that centralize all the performance information in one document is an approach used by other federal departments that could be applied in relation to the Sub-Program.

4.2.3 Gender-Based Analysis Plus

The CSA has adopted a framework that will guide the GBA Plus analysis of its programs and activities, including the Sub-Program. Moving forward, the Sub-Program will be in a position to implement such analysis, which is particularly relevant, considering the nature of its activities.

Just as this evaluation got underway, in , the federal government released its new Policy on Results, along with its associated Directive on Results, which replaced the Policy on Evaluation. This new framework clarifies expectations related to gender-based analysis. First, it confirms that in establishing their performance measurement strategy, program managers must include, where relevant, a gender-based analysis (Treasury Board of Canada Secretariat, , para. A.2.5.10). It also identifies, as a mandatory procedure, that all evaluations be planned to take into account, where relevant, gender-based analysis (Treasury Board of Canada Secretariat, , para. C.2.2.1.6).

These new requirements reflect a desire on the part of the federal government to achieve greater results on the issue of gender consideration. Back in , the federal government made a formal commitment to integrate gender-based analysis in the development of its policies, programs, and legislation (Status of Women Canada, 2016b). However, significant shortcomings in implementing this commitment continued to be identified, including those documented in the report from the Auditor General that focussed specifically on this issue (Office of the Auditor General of Canada, ). In response to this audit, the federal government released its Action Plan on Gender-based Analysis (), which includes, among other items, a commitment by the Treasury Board of Canada Secretariat to provide greater assistance to departmental evaluation functions to take into consideration Gender-Based Analysis Plus (GBA Plus) when evaluating federal programs (Status of Women Canada, 2016a).

During the period covered by this evaluation, the CSA did not have a formal gender-based analysis framework in place. As such, the goal of the evaluation is not to assess the extent to which proper gender-based analysis had been performed in relation to the Sub-Program. Instead, the objective is to assess the extent to which the Sub-Program will be in a position going forward to undertake such an analysis, as required.

Two main evaluation findings emerged on the issue of gender considerations. First, as documented in this report, the engagement of young girls in STEM-related studies, and of women in the space sector more generally, remains limited, which has triggered a number of initiatives in Canada (as well as in other countries) that specifically aim to address these shortcomings. This is the broader policy context in which the Sub-Program will continue to evolve, and which confirms that gender considerations do apply to the Sub-Program, particularly to its SAD component, and the outreach activities targeting Canadian youth.

Secondly, during the time this evaluation was taking place, the CSA was developing its own policy and procedures governing GBA Plus, which were approved in . The set of tools developed includes an implementation guide, along with a questionnaire that will help program managers to be in a position to undertake this analysis and to ensure that proper gender considerations are applied in the design and management of their programs and activities.

The ongoing performance measurement activities, along with the next formal evaluation of the Sub-Program, will provide opportunities to assess the extent to which the Sub-Program has succeeded in implementing this requirement.

4.3 Performance — Effectiveness

This sub-section of the report provides a summary of evaluation findings related to the achievement of the Sub-Program's expected results. As each component of the Sub-Program is quite distinct, component-specific results are presented.

4.3.1 SAD component

The SAD component has successfully supported the implementation of over 50 research projects, using predominantly FAST AOs. These projects have engaged faculty members and students (undergraduate and graduate) in hands-on projects that further, among other things, SST knowledge, technological development, and partnerships.

Funded projects

In terms of solicited projects, during the period covered by the evaluation, the CSA issued four AOs related to the SAD component of the Sub-Program. As indicated in Table 9, a total of 52 projects were selected through these processes, for a total of $19.1 million in funding support.

Table 9: Solicited projects per type of funding provided ()
Type of funding provided AO publication date # of projects funded Total funding provided
Cluster AO 10 $4,469,557
FAST AOTable note 11 13 $6,667,893
FAST AO 11 $2,023,995
FAST AO 18 $5,927,234
Total   52 $19,088,679

Source: CSA, Unitas data

In addition, a total of 11 unsolicited projects were approved between and , representing a total investment of $3,044,317.

In both cases (solicited and unsolicited projects), some projects approved prior to were still active during the period covered by the evaluation.

Project results and impacts

Evaluation findings provided a number of insights on the results achieved by the projects funded under the SAD component of the Sub-Program. Information and data from reports provided by funding recipients from the interviews conducted as part of this evaluation and from the case study on SAD are summarized in the following paragraphs.

While it must be appropriately contextualized and used with some caution, the information presented in Figure 6 provides an illustration of the reach that these funded projects had in . The need to be somewhat careful in using this data comes from the fact that only one year is presented in this figure (for illustrative purposes), that each reported individual may have had various levels of involvement in these projects, and that it is possible for one individual to have been involved in more than one project. With this in mind, the data indicate that a fair balance is achieved between the involvement of university professors and students. Also, the data show that undergraduate and graduate students are equally engaged in these funded projects. Participants were associated with universities (in Canada and abroad), research centres, departments and agencies, and private-sector industries.

Figure 6: Number of individuals involved in SAD funded projects in 2015-16
Research
chair/
industrial
chair
Faculty
members
Research
associate
Post-doctoral
fellows
Graduate
students
Undergraduate
students
Other
students
Number of individuals 14 109 34 35 140 215 32

(Source: Unitas data, based on 26 projects)

Of particular importance for the purpose of this evaluation is the perceived impact that these projects have had on advancing SST knowledge and allowing space partners to collaborate. As illustrated in Figure 7, the vast majority of projects reported a direct contribution to the establishment or maintenance of research partnerships, and to the ability of funded recipients to carry out outreach activities that enhanced general scientific awareness. The majority of funded projects have also led to technological or scientific breakthroughs and the development of new ideas that might be integrated in future space missions. Other significant contributions include the ability to leverage funds and to engage new players in space research. More than half of the projects funded contributed to the development of applications and algorithms and have used satellite data. In a minority of cases, the projects directly contributed to advancing the TRL of the associated technology. Finally, Figure 7 shows that only a small portion of funded projects lead to commercial successes.

Figure 7: Outcomes reported by funding recipients (2012-13 to 2015-16)
Outcome (%)
Partnership 95%
Outreach/awareness 82%
Tech. or scient. breakthrough 78%
New ideas for future missions 70%
Fund leveraging 61%
New players in space research 60%
App & algorithms development 59%
Satellite data used 57%
Multidisciplinary research 56%
Higher level of TRL achieved 40%
Commercial success 12%

(Source: Unitas data, based on 92 projectsTable note 12)

These trends were also reflected in interview findings gathered as part of this evaluation. The establishment of partnerships is seen as an important contribution of the SAD projects. As one key informant put it, "FAST and STRATOS have both been very helpful for developing partnerships. FAST has really been useful for me to partner with researchers in Canada as well as internationally." And these partnerships extend beyond Canada. As another key informant noted, "The FAST grant is allowing us to collaborate with a team at NASA and some US universities. We are taking advantage of their heavily operational study and getting science out of it."

It should also be noted that over 90% of funding recipients indicated in their final reports that they expect the partnerships they established as a result of the CSA funding to continue when their project has been completed, and that other follow-up projects will be generated as a result of this funding.

The case study report on SAD illustrates how FAST funding can lead to technological advancements. As previously noted, in this particular project relating to airborne quantum key distribution, the research team succeeded in increasing the TRL of key-related technologies. It also established partnerships with research institutions and private-sector industries, and leveraged in-kind and financial supports from five organizations. Finally, the project provides a good illustration of the range of students that these projects may engage. In this case, four post-doctoral fellows, two PhD students, two master's students, and four undergraduate students participated in the project.

On the specific impact that these projects have on students, evaluation findings indicate that both soft and hard skills are gained through SAD-funded projects. As illustrated in Figure 8, communication skills, team work, increased self-confidence, leadership abilities, and learning to adapt to change are among the skills that are reported more systematically by funding recipients. Moreover, the majority of projects allow participating students to enhance their knowledge of science, including practical experience with space science and technology. Only a minority of projects have reported a significant contribution in building system engineering experience. Overall, funding recipients were of the opinion that the SAD projects in which students were engaged will contribute in preparing them for careers in the space sector as HQP.

Figure 8: Percentage of projects that reported a significant contribution to student capacity building in selected skill areas (2012-13 to 2015-16)
Percentage
Greater science skills 97%
Introduction to SST 95%
Communication 90%
Team work 90%
Self-confidence 87%
Concrete experience with SST 82%
Capacity to adapt to change 78%
Leadership 71%
Ethical conduct 60%
Project management 51%
System engineering 27%

(Source: Unitas data, based on 92 projectsTable note 13)

Administrative data indicate that results similar to those presented in Figure 8 are also reported for projects engaging students who are financially supported by the Space Exploration and Space Utilization branches, such as the initiatives described in sub-section 4.1.1.4.

One prime indicator of the impact of funded projects on advancing space science is the level of activities in publications and other communication activities supporting a sharing of knowledge. As illustrated in Figure 9, peer-reviewed publications are the prime targets of funded recipients. Recipients have also engaged in non-peered reviewed publications and other technical publications.

In addition to publishing, funding recipients also carry out various presentations, including workshops, general public presentations, and media activities. During the period covered by the evaluation, more than 2,600 of such presentations were reported by funding recipients.

Figure 9: Number of publication producted, based on funded projets (2012-13 to 2015-16)
Peer-reviewed
publications
Non-peer-
reviewed pub.
Research
or
technical
reports
Books
(or
chapters
of books)
Reference/
training
material
Internet
publishing
Other
Number of publication 998 258 65 62 55 52 98

(Source: Unitas data, based on 85 projects)

The SAD case study provides a good illustration of the range of communication activities in which project recipients engage. As a result of the technological achievements resulting from their funded projects, participants were involved in a total of eight publications, including three in peer-reviewed journals, along with 17 presentations. Along the same line, the team engaged in outreach activities such as a laboratory tour open to the public at the Institute of Quantum Computing at Waterloo University, where they were given an opportunity to present their projects.

Evaluation findings also provided an opportunity to better understand the range of challenges that funding recipients face when they implement their projects. In doing so, and as noted during interviews, it must first be emphasized that space science and technology is a particularly complex field, where demonstration activities are bound to experience some level of challenge and technical difficulty. This is a logical condition of undertaking activities in a pre-space or space environment.

Keeping this in mind, Figure 10 confirms that some technical or scientific issues have been faced by half of the project recipients. Slightly more than one-third of them also experienced equipment or facility problems.

What may come as more of an unexpected issue is the fact that half of the funding recipients also experienced staffing issues, and a quarter of them reported funding difficulties. While the available data do not provide details on the range of staffing problems encountered, interviews conducted as part of this evaluation provide relevant insights. Since there is typically a gap of at least two years or more between AOs (depending on the space area one focusses on), interview findings indicate that it may be difficult to align these projects with the cycle of postgraduate studies. This means that students may not remain involved during the entire project period.

The same largely applies to funding. As projects are normally funded through multiple sources, interview findings indicate that it may prove difficult to align the cycle funding with the project requirements.

Figure 10: Percentage of projects that encountered issues (2012-13 to 2015-16)
Technical
or scientific
issues
Staffing
problems
Equipment
or
facility
problems
Funding
problems
Direction
of
research/
findings
Partner
interactions
Partner
withdrawal
Percentage 50% 50% 35% 25% 13% 12% 6%

(Source: Unitas data, based on 92 projectsTable note 14)

Overall, funding recipients were particularly appreciative of the support provided by the SAD component of the Sub-Program. As one key informant summed it up, "We couldn't have done any of our projects without CSA funding." The support provided is seen as providing a unique opportunity for space scientists: "The money to hire the people, the money to develop the instrumentation, and the money to execute the project are all within the FAST project. There's nothing else like it in the country."

4.3.2 STRATOS component

Through three stratospheric balloon campaigns, STRATOS allowed 15 Canadian payloads to be tested, allowing faculty members and students to be engaged in end-to-end projects involving near-space demonstration activities.

Each STRATOS campaign involves between 4 to 10 flights, each carrying a number of payloads, including, but not limited to, Canadian payloads.

Tableau 10: STRATOS campaigns ( to )
STRATOS campaign Location # of Canadian payloads tested
Evaluation period CSA facilities in Timmins 2
CSA facilities in Timmins 7
CSA facilities in Timmins 6
Post-evaluation period Esrange Space Center (Sweden) 3
Alice Springs (Australia) 6

Source: CSA, 2013b and administrative data.

As indicated in Table 10, during the period covered by the evaluation, three STRATOS campaigns were successfully completed, involving 15 Canadian payloads. Of note is the fact that the campaign was a qualification one, the first campaign to be launched from the newly established CSA Timmins Stratospheric Balloon Base. The following two campaigns were also launched from Timmins. During the post-evaluation period, in and , a total of nine Canadian payloads were also expected to participate in STRATOS campaigns, including one campaign held at the Esrange Space Centre in Sweden, and one held in Alice Spring, Australia.

While STRATOS campaigns predominantly involve teams of academics and students, program data indicate that they also involve space industries.

When exploring the impact of STRATOS, it must first be emphasized that the program operates as an enabler, as other mechanisms or funding sources must be used to develop the technology that will be eventually integrated in a stratospheric balloon flight, and to analyze the data gathered as part of these flights. As previously noted, FAST has been a predominant source of such funding. As a result, the impact of STRATOS is always linked to the impact of other activities, such as FAST.

On training HQP, evaluation findings indicate that this enabling contribution of STRATOS has a significant impact on the range of skills that funding participants, including students, can gain through this hands-on experience. The case study on STRATOS provided relevant insights in that regard. In this project, involving the ALI instrument, a total of five graduate students and eight undergraduate students participated in one of the stratospheric balloon flights. They developed, designed, and, with the close collaboration and support of the CSA's STRATOS team, tested the instrument. On two occasions, CSA engineers visited the students to review and discuss their project. The student also collaborated with an industrial partner. Findings from this project have already been used in two peer-reviewed journals, and three theses. The team also had the opportunity to present its findings at the ESA Rocket and Balloon Workshop in Norway.

This added dimension that STRATOS can bring was also acknowledged during interviews conducted as part of this evaluation. As one key informant put it:

"Our project would not have happened without CSA support. Some of the work would have been carried out, but it would have been kept at a fairly low level and we probably wouldn't have achieved the flight opportunities that we had through STRATOS. We might still have developed some instrumentation but flying the technologies almost certainly wouldn't have happened without CSA support."

Evaluation findings also indicate that STRATOS activities directly support the advance of space technologies. The project examined as part of the STRATOS case study became the first suborbital demonstration of an imaging aperture-acousto tunable filter (AOTF) for atmospheric remote sensing. It also contributed to increasing the TRL of key associated technologies. The campaign also provided an opportunity for Xiphos Systems Corporation to test and qualify its Q6 processor, which has since flown on a number of missions, including one involving the International Space Station (Xiphos Systems, ).

4.3.3 ED1 component

ED1 activities provided space engineering expertise to both internal and external stakeholders, including activities specifically targeting students involved in space-related fields of study.

As previously noted, the ED1 component of the Sub-Program provides a range of support to internal and external stakeholders, including those involved in other components of the Sub-Program, such as FAST, STRATOS, and the Junior Engineers Training Program. Sub-section 4.1.1.3 of this report has already noted the relevance of the subject-matter expertise that this group provides.

As a result of the nature of this activity, evaluation findings cannot offer an illustration of all the various contributions that ED1 has made during the period covered by the evaluation. However, one such illustration is provided by the case study report on the Canadian Satellite Design Challenge. Over the course of the period covered by the evaluation, two competition rounds were held ( and ), each lasting two years, and involving 8 to 12 interdisciplinary teams of undergraduate and graduate students. ED1 team members served as subject-matter experts and judges during these competitions, in addition to facilitating the access to the David Florida Laboratory (DFL) testing facilities.

The contribution of ED1 to HQP development is also illustrated through its support to the master's degree program offered by the Comité sectoriel de main-d'œuvre en aérospatiale du Québec (CAMAQ), which involves five universities in Québec (McGill, École de technologie supérieure Montréal, the Polytechnique, Concordia, and Sherbrooke). Every second year, members of the ED1 group deliver one of the eligible courses (Aerospace Case Studies), which covers the fundamentals in space mission and subsystem design.

4.3.4 Junior Engineers Training Program

The Junior Engineers Training Program provides a unique opportunity for emerging engineers to join the space workforce.

Evaluation findings confirm the positive impact of the Junior Engineers Training Program on its participants. As noted in the discussion on the relevance of the component (see sub-section 4.1.1.3 of this report), young engineers who were provided with the opportunity to join the program considered it an exceptional opportunity.

The case study focussing on this program also provides additional insights on the results achieved. Of note is the fact that, of the eight individuals who took part in the first recruitment campaign, seven have successfully completed the program, including six who have now joined the CSA as engineers. Those recruited during the campaign were still completing the program at the time of the evaluation. Program participants are also provided with the opportunity to engage in outreach activities targeting Canadian youth, and to teach courses at the graduate level.

4.4 Performance — Efficiency and economy

During the evaluation period, the Sub-Program experienced significant fluctuations in its budget, which were reflected in the range of activities it undertook, particularly through the SAD component. At the time of the evaluation, the CSA was implementing processes to enhance the integration, coordination, and planning of the various components of the Sub-Program.

This last sub-section of the report summarizes evaluation findings related to the efficiency of the program delivery.

4.4.1 Resource allocations

The Sub-Program experienced substantial shifts in human and financial resources throughout the period covered by the evaluation, and a number of these changes occurred as a result of the implementation of the program review. This is important contextual information when it comes to analyzing the efficiency of the resource allocation process.

The largest portion of the Sub-Program's financial resources was allocated to the SAD component. As previously indicated in Table 1 (page 6), the actual expenditures of this Sub-Component went from $11.8 million in down to $3.4 million in . In total, the expenditures for this component were $1.4 million less than initially planned by the CSA.

The second largest investment of the Sub-Program was directed to ED1 activities. In this case, and as previously indicated in Table 2 (page 7), the total budget went from $4 million in , down to $1 million in . In total, the expenditures for this component were $3.4 million less than initially planned.

In both cases, and as illustrated in Figure 11, the number of FTEs assigned dropped significantly during the evaluation period, as a result of restructuring (including the dissolution of the Space Science directorate) and reallocation of resources from ED1 to STRATOS.

Figure 11: Actual FTEs assigned to SAD and ED1 activities per fiscal year
- - - - -
SAD 16.1 12.8 5.1 2.6 1.8
ED1 21.8 24.3 9.3 7.1 5.7

(Source: financial data)

As for the STRATOS program, close to two-thirds of the actual expenditures incurred during the period covered by the evaluation were assigned to operations and maintenance, which were essentially used to establish the Stratospheric Balloon Base in Timmins. Out of a total of $9.1 million in actual expenditures over five years, $5.8 million was allocated for this purpose.

As for the Junior Engineers Training Program, resource allocations were linked to actual recruitment campaigns and were mostly allocated to cover the salaries of program participants. In the case of outreach activities targeting Canadian youth (Space Learning, as well as Awareness and Learning), their associated expenditures essentially ceased in .

Interview findings indicate that this environment has been particularly challenging for those involved in the delivery of the Sub-Program activities. The uncertainty and the decreasing capacity to carry out all expected activities covered by the Sub-Program have raised concerns as to the ongoing sustainability of the Sub-Program to achieve its expected results. Evaluation findings indicate that the SAD group is experiencing challenges with regards to its capacity to monitor all current projects, offer the required support to funding recipients, manage unsolicited proposals, and appropriately plan for future AOs. In this context, it would not appear reasonable to expect that the CSA provides — if such was its desire — more predictable AOs as recommended by many of the participating stakeholders. Along the same line, and as already noted, the current capacity level of the ED1 group is significantly narrower, and it remains unclear how the expertise currently provided will be sustainable in the long term.

4.4.2 Processes to enhance efficiencies

At the time of the evaluation, a number of processes were in place to enhance the efficiency of the Sub-Program, including the following components:

  • Through the establishment of the Science, Technology and Expertise Development in Academia (STEDIA), which replaces the SAD, the CSA wishes to enhance its ability to coordinate and integrate all activities related to the development of HQP and the growth of SST knowledge.
  • The CSA was also in the process of centralizing and upgrading all of its support for demonstration activities, ensuring that STRATOS and other demonstration avenues were being implemented in a coordinated manner. The CSA has set aside a total of $18.4 million over five years ( to ) for capacity demonstration activities.
  • The CSA has also implemented processes to enhance the planning, monitoring, and implementation of activities falling within the ED1 component. While this component remains committed to operate on a matrix basis, and being sufficiently flexible to respond to the various needs of internal and external stakeholders, the goal is to ensure that these activities remain aligned with priorities of the agency.

These measures are expected to support the CSA as it engages in the various commitments contained in its Departmental Plan that relate to the development of space expertise and space capability demonstration (Canadian Space Agency, 2017a, p. 21).

Otherwise, evaluation findings have not identified alternative mechanisms that could be used to more effectively deliver the support provided through the Sub-Program.

5. Conclusions and recommendations

This section of the report concludes by summarizing the key findings of the evaluation, and incudes observations and recommendations where applicable. More detailed information, substantiating each of these statements, is included in the preceding sections of the report.

5.1 Relevance

In terms of relevance, the Space Expertise and Proficiency Sub-Program is consistent with federal roles and responsibilities, and aligns with the CSA and federal priorities to develop HQP, advance SST knowledge, and promote science, technology, engineering, and mathematics (STEM). Overall, the evaluation indicates that there is a need for the Sub-Program to continue supporting Canadian space capacity development.

SAD

The evaluation indicates that the SAD component of the Sub-Program responds to the need of Canadian research institutions to engage in hands-on space research projects. SAD activities represent a significant means by which the CSA supports SST knowledge growth, including the development of HQP in Canada. Research institutions require opportunities to engage students in SST projects in order to provide adequate training. In turn, projects advancing SST require the participation of students to support their implementation. The SAD component of the Sub-Program provides such opportunities and while other granting organizations play a critical and complementary role, what the SAD component offers has no equivalent in Canada.

During the five years covered by the evaluation ( to ), the SAD component of the Sub-Program was largely focussed on FAST funding, which responds principally to the need for end-to-end projects. Only limited activities were undertaken in other priority areas identified by research institutions such as research clusters, industrial chairs, and other studies on space science instruments or data analysis. Responding to the needs of research institutions to further undertake such research activities thus represents an opportunity for improving the SAD component of the Sub-Program.

STRATOS

The evaluation demonstrates that having access to stratospheric balloons in Canada represents a well-suited tool to support SST and HQP development. The STRATOS component is particularly well-suited for supporting end-to-end research projects, which is the focus of FAST funding. In that sense, these two types of activities are highly complementary. The relevance of the stratospheric balloon activities is also illustrated by the fact that it is a fairly flexible, accessible, and efficient demonstration option.

Broadening the range of suborbital platform and stratospheric balloon activities (in terms of payload weight and flight duration), and integrating in a more systematic fashion the support provided for other types of demonstration activities would further enhance the capacity of research institutions to engage in SST and HQP development.

ED1

The group of experts under the ED1 component provides valuable expertise to both internal and external stakeholders. In addition to providing needed matrix support for internal CSA funding initiatives, including SAD and STRATOS, ED1 activities contribute to the CSA's participation in a number of space-related international fora.

ED1 members also support HQP development by, among other things, acting as external examiners or co-supervisors of theses, and by supporting engineering capstone projects. While many benefits can be derived from having CSA engineering experts involved with students activities, it would be beneficial to clarify the rationale for doing so, and the specific niche that the CSA wishes to occupy in that regard.

Junior Engineers Training Program

During the period covered by the evaluation, a number of emerging engineers have joined the CSA through the Junior Engineers Training Program, exposing them to the full range of activities undertaken by the Agency's various directorates. As such, it represents a remarkable opportunity to integrate new HQP to the space sector. In turn, the Junior Engineers Training Program provides a valued opportunity to join the ranks of the CSA, while the program is also contributing, to some extent, to renewing CSA's internal engineering expertise.

Awareness and Learning and Space Learning

The evaluation describes the wide range of activities undertaken by the CSA, as well as in Canada and around the world to engage youth (elementary and secondary students) and post-secondary students (college and undergraduate students) in STEM. There is a strong rationale to pursue these activities in order to reverse the documented decline in STEM interest, and the lower level of STEM engagement found among young women.

As the evaluation demonstrates, the CSA possesses a unique attraction pole to engage young Canadians in STEM fields of studies to which the Sub-Program contributes. The CSA holds unique assets that it can lend to this important endeavour, through the involvement of the Sub-Program and its other directorates. At the time of the evaluation, the Communications and Public Affairs Directorate was carrying out a wide range of outreach activities targeting Canadian youth, and the Sub-Program's contribution was more focussed on supporting specific activities for university students (such as attending space-related conferences or through projects funded by the SAD component). The CSA is further structuring these activities through a STEM plan that is expected to cover activities targeting both Canadian youth and students, and through revisions to the component of its Class G&C Program relating to Awareness and Learning.

5.2 Design and delivery

Overall, the Sub-Program components' design and delivery are sound and the CSA has established efficient structures and processes to manage the Sub-Program. The evaluation has identified opportunities for improvement, or confirmed the relevance of current initiatives undertaken by the Sub-Program for improving its design and delivery in order to enhance the Sub-Program's ability to achieve its expected results.

The SAD component delivery structure is sound in its design. However, the level of internal resources (FTEs and funding) currently allocated for the delivery and management of the SAD program has been drastically reduced during the period covered by the evaluation. It is doubtful that this structure would be in a position to accommodate more regular AOs, as advocated throughout the interviews held as part of this evaluation, or any additional requirements that may emerge. Concerns were raised as per the ability of the SAD management team to monitor current projects, or to capitalize on ad hoc opportunities offered by other space partners.

The STRATOS component design is sound, as the CSA has established an efficient structure to manage STRATOS activities and its partnership with the CNES. While stratospheric balloon flights constitute an efficient avenue to test and advance new technologies, other forms of demonstration activities are also required to accommodate the range of space technologies in which Canadian space partners are engaged. The STRATOS activities would benefit from being more integrated within the range of demonstration activities in which the CSA is involved.The CSA is currently developing a framework to better integrate these demonstration activities, and this evaluation confirms the relevance of this process.

As for internal activities covered by the Junior Engineers Training Program and ED1, the CSA has established processes to adequately support their management. The CSA has initiated a process to better plan and monitor its internal activities, particularly those related to ED1. The new framework being used to plan and monitor ED1 activities will contribute to a better understanding of the range of activities this group undertakes, and how they relate to the goals and priorities of the CSA. However, just as it is the case with the SAD group, it remains unclear how the CSA will sustain the level of expertise currently offered by this group in the long term.

The evaluation has noted that the Sub-Program has developed a performance measurement strategy that adequately covers the range of activities it undertakes, and their associated outcomes. What has yet to be clarified is how the CSA will collect, store, and share the performance information identified in the strategy. The work of the CSA in implementing the performance measurement reporting requirements included in the new Treasury Board of Canada's Policy on Results should provide an opportunity to address this issue.

Finally, the evaluation has noted that the CSA has developed a policy and a set of tools that will allow the Sub-Program to carry out gender-based analyses to ensure that the planning and delivery of its activities can reach all expected stakeholders, as defined in GBA Plus. Evaluation findings leave no doubt as to the importance, relevance, and applicability of gender considerations in relation the Sub-Program.

5.3 Performance

The Sub-Program's activities have contributed to the achievement of its expected outcomes, but a full assessment of these outcomes must also consider a range of other activities undertaken by the CSA.

Immediate outcome 1: Partnerships among research organizations established or maintained

During the period covered by the evaluation, the Sub-Program contributed to the establishment of a wide range of valuable partnerships, including those involving universities and research centres, those that enabled collaborations between universities and space industries, and those collaborations reaching out to other space agencies or space organizations. Activities under FAST, STRATOS, or ED1 have all supported various forms of partnerships, with a predominant number relating to end-to-end projects. In a number of cases, these partnerships have benefitted from funding support provided by other granting organizations, such as NSERC.

Immediate outcome 2: Increased or maintained number of students and HQP in SST

The evaluation demonstrates that the Sub-Program has allowed students and HQP to be involved in SST projects. Sub-Program activities have facilitated the involvement of students, both undergraduate and graduate students, in a wide range of activities, with a particular focus on hands-on projects such as those under FAST and STRATOS. These projects add a new and important dimension to the education students receive through their study programs.

The Junior Engineers Training Program has also provided a unique opportunity for those selected participants to successfully engage in SST, by joining the CSA workforce.

Immediate outcome 3: Increased or maintained number of research projects performed in SST

The Sub-Program has proven to be a key contributor, along with other granting organizations in Canada, to research projects performed in SST. These projects have supported the growth of SST knowledge in a number of areas targeted by the CSA, particularly through its FAST AOs. This has, in turn, contributed directly to the fundamental and legislative object of the CSA to "advance the knowledge of space through science."

Immediate outcome 4: Increased or maintained possibilities of accessing pre-space and space

The Sub-Program's direct contribution to accessing pre-space and space environments has been focussed on stratospheric balloon launches. In collaboration with the CNES, the Sub-Program has successfully launched stratospheric balloon campaigns throughout the period covered by the evaluation, including campaign launched from the newly established CSA Timmins Stratospheric Balloon Base. These campaigns have predominantly engaged students and academics, but also the space industry. Funding provided by components such as FAST AOs also provided access to other demonstration opportunities, such as analog sites and parabolic flights.

Notwithstanding the achievement by the Sub-Program of its expected outcomes, the evaluation indicates that transitioning space-focussed students and HQP into the labour market remains a concern. While some of the students benefitting from the Sub-Program's activities will be able to pursue a space career in Canada in their field of expertise, evaluation findings indicate that others will need to apply generic skill sets in other space-related fields or apply these skills in other STEM-related fields that are not directly related to space. In this context, the implementation of the Sub-Program activities should be done in such a way as to support, when feasible, the transition of the trained HQP in various space-related fields, including space science and technology, space exploration and space utilization, and in non-space fields.

In order to address the concern raised above, the evaluation recommends the following:

The Space Science and Technology Directorate ensures that its objectives and activities are aligned not only with current but also emerging needs of Canadian universities and research institutions in advancing space science and technology knowledge, while also facilitating the work transition of the developed HQP toward sectors, including but not limited to the Canadian space sector, where their advanced technical skills and behavioural competencies (soft skills) can be best applied.

This recommendation is based on the premise that the range of skills developed through the projects funded by the Sub-Program will directly match some of the HQP requirements of the space sector, including the space industry, while acknowledging that the challenge related to transitioning toward the labour market far exceeds the strict parameters of the Sub-Program.

Finally, the evaluation provided an opportunity to document how other activities in which the CSA is engaged contribute to the development of space-related HQP and SST. The goal was not to formally evaluate these activities, but rather to explore their relationship with the Sub-Program, and their potential contribution to the achievement of the Sub-Program's expected outcomes. Activities undertaken by the CSA beyond the Sub-Program also contribute to these expected outcomes. Both the Space Exploration and Space Utilization branches engage universities and research institutions in a range of projects that, in a complementary manner, also further SST knowledge and train HQP.

Along the same lines, the evaluation provided an opportunity to better situate the work of the CSA in promoting STEM studies among Canadian youth, by documenting not only what is already done by the CSA (within and outside of the Sub-Program), but also how a wide range of other stakeholders contribute to this goal.

This analytical approach has provided a wider context that has supported a better and more integrated understanding of the Sub-Program activities and achievements.

Management Response and Action Plan

Responsibility
Organization / Function
Management response Details of action plan Schedule
Recommendation # 1
The Space Science and Technology directorate should ensure that its objectives and activities are aligned not only with current but also emerging needs of Canadian universities and research institutions in advancing space science and technology knowledge, while also facilitating the work transition of the developed HQP toward sectors, including but not limited to the Canadian space sector, where their advanced technical skills and behavioural competencies (soft skills) can be best applied. DG Space Science and Technology (SS&T)

SS&T accepts the recommendation. It will be taken into account while implementing SS&T-led initiatives in the context of the new Departmental Results Framework (DRF).

Measures to encourage involvement of students in Space Technology Development Program (STDP)-related activities were recently implemented.

  1. Pursue formal engagement with interested stakeholders, especially academia and industry, as Science and Technology Expertise Development in Academia (STEDIA) and other funding initiatives are being developed or modernized.
  2. Collaborate with organizations such as MITAC to increase student internships in industry;
  3. Undertake an analysis (or study) to better understand workforce needs of industry and target training efforts accordingly.

Appendices

Appendix A: Bibliography

Date modified: