Evaluation of the Space Astronomy Missions and Planetary Missions Programs
From the period from to
Project # 16/17 02 - 02
Prepared by the Audit and Evaluation Directorate
Table of contents
- List of tables and figures
- Acronyms used in the report
- Executive summary
- Evaluation approach and methods
- 3.1 Purpose and scope
- 3.2 Evaluation issues and questions
- 3.3 Methods
- 3.3.1 Documentation review
- 3.3.2 Key informant interviews
- 3.3.3 E-surveys
- 3.3.4 Case studies
- 3.4 Limitations
- 4.1 Relevance (section 4. Findings)
- 4.2 Effectiveness (section 4. Findings)
- 4.2.1 Outputs
- 4.2.2 Immediate outcomes
- 4.2.3 Intermediate outcomes
- 4.2.4 Ultimate outcomes
- 4.3 Efficiency (section 4. Findings)
- Conclusions and recommendation
- Management response and action plan
List of tables and figures
- Table 1: Resources allocated to the SAM & PM programs for the evaluation period
- Table 2: SAM annual overhead rates, –
- Table 3: PM annual overhead rates, –
- Table 4: SAM & PM combined annual overhead rates, –
- Figure 1: SAM & PM Missions by life-cycle phase, –
- Figure 2: SAM Planned vs Actual disbursements in $1,000s, –
- Figure 3: PM Planned vs Actual disbursements in $1,000s, –
- Figure 4: Distribution of total disbursements in $1,000s, –
Acronyms used in the report
- APXS: Alpha Particle X-Ray Spectrometer
- CAMS: Canadian ASTRO-H Metrology System
- CSA: Canadian Space Agency
- ESA: European Space Agency
- FGS: Fine Guidance Sensor
- FTE: Full-Time Equivalent
- G&C: Grants and Contributions
- GTO: Guaranteed Time Observation
- HFI: High Frequency Instrument
- HIFI: Heterodyne Instrument for the Far Infrared
- HQP: Highly Qualified Personnel
- JAXA: Japan Aerospace Exploration Agency
- JWST: James Webb Space Telescope
- LFI: Low Frequency Instrument
- LiDAR: Light Detection and Ranging
- MATMOS: Mars Atmospheric Trace Molecule Occultation Spectrometer
- MDA: MacDonald, Dettwiler and Associates
- MOU: Memorandum of Understanding
- MSL: Mars Science Lab
- NASA: National Aeronautics and Space Administration
- NIRISS: Near Infrared Imager and Slitless Spectrograph
- NRC: National Research Council Canada
- NSERC: Natural Sciences Research and Engineering Council of Canada
- Oc: Outcome
- OLA: OSIRIS-REx Laser Altimeter
- Op: Output
- O&M: Operations and Maintenance
- PI: Principal Investigator
- SAM & PM: Space Astronomy Missions & Planetary Missions
- SPIRE: Spectral and Photometric Imaging Receiver
- SSP: Sub-Sub Programs
- UVIT: Ultraviolet Imaging Telescopes
The Evaluation of the Space Astronomy Missions (SAM) and Planetary Missions (PM) programs of the Canadian Space Agency (CSA) was commissioned in by the Audit and Evaluation Directorate in accordance with the requirements of the Financial Administration Act and the Five-Year Departmental Evaluation Plan. The evaluation undertaken by Science-Metrix Inc. was conducted in accordance with the Treasury Board of Canada's Policy on Results and addressed the evaluation issues of relevance, effectiveness and efficiency. The present report constitutes the first evaluation of the SAM & PM programs. The timeframe of the evaluation covered the period from to during which time total disbursements amounted to over $110 million.
The CSA has been carrying out activities geared toward Canadian participation in space astronomy since and in planetary missions since . SAM encompasses the definition, design, technology development, implementation and use of complete Canadian space telescope systems and the provision of Canadian instruments, sensors and subsystems to international space telescope or probe missions. It generates scientific data about the universe through the observation of the solar system and deep space. PM encompasses the definition, design, technology development, implementation and use of Canadian exploration signature technologies and scientific instruments made available to international exploration missions. It supports the robotic exploration of remote bodies, such as planets and asteroids, to conduct detailed observations and science.
In carrying out this evaluation, a participatory approach was employed that entailed the use of mixed quantitative and qualitative methods. Specifically, document and archival data reviews were performed, e-surveys were administered to 41 funding recipients, and key informant interviews were conducted with 24 CSA representatives, one other government department, and 3 international partners. Two case studies were completed, one for each of the two programs under evaluation. The focus of these case studies was Canada's contribution to NASA's James Webb Space Telescope (JWST) mission and the Mars Science Lab (MSL)/Curiosity rover mission.
As a whole, the SAM & PM programs remain relevant and continue to meet the needs of the Canadian scientific community. With relatively modest investments, the SAM & PM programs were successful in obtaining access to space mission infrastructure that Canada on its own could never afford, such with JWST and the MSL. In exchange for these relatively modest investments, there were scientific, cultural and economic benefits for Canadians. It was also through this program that the Microvariability and Oscillations of STars telescope (MOST) was developed, the first CSA-led space astronomy mission in the history of Canada's space program.
The SAM & PM programs are well aligned with the government's innovation agenda and the CSA's mandate and strategic outcome. The SAM & PM programs are essential to maintaining a world-class cadre of astronomers and planetary scientists in Canada dedicated to advancing the knowledge of space through scientific discovery. The current space astronomy and planetary missions that are in the development and operational phases will continue to meet the needs of their respective scientific communities with opportunities to conduct space exploration research, access scientific data and contribute to the advancement of knowledge for several years. The evaluation found that the SAM & PM programs are highly relevant to the CSA mandate and its strategic outcome as they develop Canada's space exploration capacities to advance the knowledge of space through science and technological innovation. In addition, the CSA is the only federal organization that provides the scientific community with access to space astronomy opportunities and data.
The SAM & PM programs are very cost-effective and have achieved notable accomplishments which have contributed to Canada's reputation in the international space exploration community. Canada's participation in international partner space missions is contingent on making a contribution in the form of a science instrument, subsystem or related component. The space astronomy and planetary missions that Canada has led, such as MOST, or more often contributed to, included one or more science instruments and/or subsystems. Throughout the period of the evaluation, the SAM & PM programs have developed or operated 6 science instruments and 4 subsystems or spacecrafts. By , SAM & PM will meet or exceed their targets with two science instruments for space astronomy and planetary missions respectively, delivering data on a regular basis to the Canadian scientific community. The evaluation found that, at times, the engagement of the science teams was delayed due to a lack of grant funding at start-up and fragmented funding for scientific research during and after missions. The program should develop clear guidelines that clarify the definition and scope of science support to missions in order to allow for continuity in science support funding throughout all phases of a mission from pre-definition phase to post-operation activities.
Canada's success in space exploration missions is contingent on collaboration between industry, academia and government. The achievement of significant outcomes of the JWST and MSL missions, where government, academia and space industry partners worked collaboratively to deliver high-performance scientific instruments (NIRISS, APXS) and critical subsystems (FGS) are but a few examples of how Canada's international profile and reputation have been enhanced with each successfully completed mission deliverable. This has also resulted in a Canadian space exploration sector that is better positioned to seize space opportunities, and reuse its know-how and technology in future space missions to generate scientific discoveries.
The evaluation findings are unequivocal with respect to the positive contribution that the SAM & PM missions have made to Canada's space exploration profile and reputation. The combination of space-tested technologies, scientific and engineering know-how, and can-do attitude are often-noted characteristics of Canada's space exploration sector. Recognized as a trusted and reliable NASA partner, the CSA and its partners have been able to forge new partnerships to undertake space astronomy and planetary missions with the increasingly active space agencies of several other countries, including Japan and India and to export tested technologies and scientific instruments to other countries. However, the evaluation has also found that the irregular cadence of the SAM & PM programs has resulted in human and financial management challenges for industry partners and universities researchers, and uncertainty among international partners regarding Canada's financial commitments to ongoing and proposed missions. The enviable reputation that Canada has established over many years with its signature contributions to international joint ventures and high-profile space exploration missions may have been affected over the years due to the lack of investments.
The evaluation found that the program was very efficient with regards to leveraging and cost-efficiencies. The sustained data access and increased competence outcomes for the Canadian scientific community have been achieved at a very reasonable cost to the Government of Canada in comparison to the overall mission costs of its international partners. In exchange for these relatively modest investments come numerous benefits: scientific, cultural and even economic. The Canadian astronomy and planetary sciences communities have opportunities to influence the direction of the space research, design and deliver their own scientific instruments, have guaranteed time of observation, and priority access to the research data for at least six months. These advantages are not negligible and serve to maintain Canada's influential position among the leaders in space exploration research, which is a source of pride and inspiration to attract and retain highly qualified personnel in the science, technology, engineering and mathematics disciplines that are essential to an innovative and productive Canadian economy.
The mandate of the Canadian Space Agency (CSA) is "
to promote the peaceful use and development of space, to advance the knowledge of space through science and to ensure that space science and technology provide social and economic benefits for Canadians."Footnote 1 Established in , with a status equivalent to that of a Department of the Government of Canada, the CSA is responsible for the coordination and implementation of space policies and programs, the application and diffusion of space technology, and the promotion of commercial exploitation of space.
The CSA Audit and Evaluation Directorate commissioned the services of Science-Metrix to undertake an Evaluation of CSA's Space Astronomy Missions (SAM) and Planetary Missions (PM) programs as per the Five-Year Departmental Evaluation Plan and in accordance with the Treasury Board of Canada's Policy on Results.Footnote 2 The evaluation was conducted during the – fiscal year, under the direction of the CSA's Audit and Evaluation Directorate (specifically, the CSA's Evaluation function) and covers the period from to .
2.1 Program profile
The CSA has been carrying out activities geared toward Canadian participation in space astronomy since and in planetary missions since . Following restructuring of the CSA in – and the implementation of the CSA's Program Alignment Architecture in –, the SAM & PM programs have been identified as sub-sub-programs (SSPs) 18.104.22.168 and 22.214.171.124, respectively, which fall under the sub-program Exploration Missions and Technology 1.2.2, which in turn falls under the CSA's Space Exploration Program 1.2. In the interests of concision, the two sub-sub-programs under evaluation shall be referred to as programs throughout this report.
Space Astronomy Missions (SSP 126.96.36.199) encompasses the definition, design, technology development, implementation and use of complete Canadian space telescope systems and the provision of Canadian instruments, sensors and subsystems to international space telescope or probe missions. It generates scientific data about the universe through the observation of the solar system and deep space.Footnote 3
Planetary Missions (SSP 188.8.131.52) encompasses the definition, design, technology development, implementation and use of Canadian exploration signature technologies and scientific instruments made available to international exploration missions. It supports the robotic exploration of remote bodies (planets, asteroids, etc.) to conduct detailed observations and science.Footnote 4
2.2 Program theory of change
The SAM & PM logic model presented in Appendix A provides a visual representation of the means by which resources that are allocated to the programs are used to produce key outputs, leading to the achievement of expected outcomes. The following narrative drawn from the SAM & PM programs' Performance Measurement StrategyFootnote 5 describes these expected program outcomes and the program theory of change.
The SAM & PM programs have four immediate outcomes (Oc):
- Sustained Access to Scientific Data (Oc1): Space astronomy and planetary exploration science investigations and science instruments generate scientific data related to the solar system, our galaxy and the universe. In order to maximize returns, in the longer term, the data are made available to Canadian researchers by processing data and providing access to archived data or curated samples. This immediate outcome leads to increased knowledge about the Universe, solar system and human capacity to live in Space (Oc7).
- Increased Number of Highly Qualified Personnel (Oc2): The SAM & PM programs provide opportunities for individuals and organizations in the private and academic sectors to showcase their capabilities. Space astronomy and planetary missions create professional opportunities to develop and employ highly qualified personnel (HQP) in Canada, sometimes through participation in international missions. As a consequence, the number of HQP in space astronomy and planetary exploration in Canada is increased. For HQP measurement, the definition found in the State of the Canadian Space Sector report is used and entails tracking the number of employed engineers, scientists and technicians.Footnote 6 In this case, HQP will be specifically for space astronomy and planetary missions. This immediate outcome leads to a Canadian Space Exploration Sector that is better positioned to seize space opportunities (Oc5) and increased knowledge about the universe, solar system and human capacity to live in space (Oc7).
Expanded Canadian Presence in Space through Space Missions (Oc3): For this outcome, the expression "
presence in space" is used in an inclusive way, from activities in low Earth orbit, to instruments (probes) sent to solar system bodies to enable conduct of science investigations and the operation of science instruments, as well as subsystems or spacecraft. Examples of expanded Canadian presence in space for the PM program include the rover Curiosity of the Mars Science Laboratory mission, which carries a Canadian science instrument (the Alpha Particle X-ray Spectrometer), and the ExoMars rover, which will carry two Canadian subsystems contributions (Bogie Electromechanical Assembly and Navigation cameras). An example for the SAM program is the Fair Ultraviolet Spectroscopic Explorer (FUSE) telescope, which operated the Canadian-built Fine Error Sensor camera system used to stabilize and point with extreme precision.
Given the relatively small size of the CSA's space exploration program in comparison to that of other nations, Canada's preferred option is to partner with other space agencies. To ensure that such missions contribute to achieving objectives valued by the Government of Canada, the SAM & PM programs work with the Canadian space exploration sector to contribute science investigations, instruments, and subsystems and spacecraft. As a result of having demonstrated its capacity through delivery of space astronomy or planetary missions, a space sector entity is in a better position to respond to national and international mission opportunities when they arise, or to transfer technology and know-how. This immediate outcome leads to the transfer of know-how and technology to other applications (Oc5), and to seizing other space opportunities (Oc6).
- Increased Competence of Space Sector in Space Exploration Areas (Oc4): Competencies in space astronomy or planetary exploration missions are developed and maintained by industrial or academic entities in Canada. These competencies span such fields as analyzing data from space, conceiving missions, and developing or operating scientific instruments and subsystems or spacecraft. Competencies concerning the production of spacecraft or subsystems place the Canadian space exploration sector, industry and academia in a position to generate intellectual property, in the form of registered trademarks, patents, copyrights or industrial designs. This outcome may lead to transfer of know-how and technology to other applications, or to seizing other space opportunities (i.e., Oc5 and Oc6).
The SAM & PM programs have three intermediate outcomes:
- Canadian Space Exploration Sector Is Better Positioned to Seize Space Opportunities (Oc5): For this outcome, space opportunities refer to national endeavours dealing with space astronomy or planetary exploration, as well as to international missions that foreign space agencies embark on regularly. Demonstrated experience in dealing with the space environment is sought after by foreign space agencies. As the industrial or academic entities have produced spacecraft, subsystems or instruments, or as HQP have worked with space data, they are in a better position to seize opportunities that occur in space astronomy or planetary missions. This outcome leads to sustained economic growth (Oc8) and an enhanced space exploration profile in Canada and abroad (Oc9).
- Enhanced Transfer of Know-how and Technology to Other Applications (Oc6): The high level of ingenuity involved in developing and operating spacecraft, subsystems and science instruments that enable machines and humans to function in space generates innovative know-how (expertise, processes) and technology that is often applied for other purposes, either in space or on Earth. Technology can be transferred to another space device or it can be adapted and transferred to a device that will be used on Earth. Working with innovative space technologies increases the likelihood that additional applications can be found. This outcome contributes to economic growth (Oc8) and to the space exploration profile (Oc9).
- Increased Knowledge about the Universe, Solar System and Human Capacity to Live in Space (Oc7): By applying state-of-the-art technologies to realize space missions, as well as by seizing opportunities, the resulting foremost science investigations push the limits of our knowledge about the universe and the solar system. The SAM & PM programs contribute to discoveries concerning the nature and origin of the universe and solar system, and the capacity for humans to live and work in space. This outcome leads to Oc9, an enhanced space exploration profile in Canada and abroad.
The SAM & PM programs have two ultimate outcomes:
Sustained Economic Growth (Oc8): If the industrial and academic entities in the Canadian space exploration sector are better positioned to capture opportunities because of their expertise in producing spacecraft and subsystems, they can generate additional business. The industrial or academic entities are then in a position to gain more contracts, nationally or internationally.
Some of the products or technologies initially transferred to new applications on a small scale may eventually prove to be so useful that they become commercially viable and make significant contributions to the Canadian economy. Foreign space agencies have documented many examples of products, processes and technologies developed for space exploration that have been adapted for non-space consumer products and services. Oc5 contributes to this ultimate outcome because as opportunities are grasped, new avenues are available for new participants to expand the economic sphere. Transfer of know-how and technology to other applications (Oc6) also contributes to this ultimate outcome because adaptations from space use to non-space use (new consumer products) contribute to job creation in the industry or academic sectors and generate economic growth in general.
- Enhanced Space Exploration Profile in Canada and Abroad (Oc9): In the context of this ultimate outcome, the space exploration profile has three broad components:
- citizens' involvement,
- scientific results, and
- geopolitical impacts (bilateral and multilateral collaborations and partnerships with other countries).
It is generally recognized that space exploration, more specifically space astronomy and planetary exploration, has the capacity to inspire citizens by opening up new horizons, literally and figuratively. One way to achieve that is through the popularization of knowledge gained from science investigations. That knowledge, which has been reflected in important journal publications (Oc7), will be presented in popular magazines and other media accessible to a broader public. The wonders of exploration, demonstrated among other things by the Canadian presence in space through space missions (Oc3), may pique Canadians' curiosity and bring them to seek more information about astronomy or planetary exploration through newspaper articles, webpages or social media. Enhancing the profile in Canada refers to how Canadians react to the different activities and missions taking place in space when Canadian competence is demonstrated after providing functional subsystems or instruments (Oc4).
The profile of space exploration can be enhanced by the scientific results (Oc1, Oc2 and Oc7) delivered by planetary rovers, space telescopes or other science instruments. Though Canada's funding in exploration research is modest compared to other countries, its world-class expertise is sought by those same countries and their respective space agencies. The quality of the Canadian scientific results is demonstrated by leading organizations (such as the Organisation for Economic Cooperation and Development) that reference the data and information. The perception on the international scene of Canadian research in space astronomy and planetary missions enhances the profile of space exploration.
The profile of space exploration can also be enhanced by the realization that cooperation is very often needed between countries as they pursue very expensive endeavours. That cooperation, related to scientific or technological domains (Oc5, Oc6 or Oc7), is not simply done from agency to agency but requires that governments be involved. The International Space Station is a good example of such cooperation taking place at government level. More specifically, the Canadian contributions to the James Webb Space Telescope or to the Curiosity rover have also required that cooperation between companies and universities be framed by governmental agreements. Participation in international space astronomy or planetary missions may help position Canada as a dependable partner, enhancing international relations on many fronts and branding Canada as an innovative, forward-looking nation. Thus the space astronomy and planetary mission outcomes (Oc3, Oc5, Oc6 and Oc7) are aligned with enhancing Canada's national and international profile in research and engineering activities.
2.3 Governance and roles and responsibilities
The Director General of Space Exploration is accountable to the President of the CSA and chairs the Space Exploration Management Committee in order to ensure effective coordination of all financial or human resources across the sub-programs under his authority. The Director, Space Exploration Development is responsible for carrying out the decisions taken by the Space Exploration Management Committee that pertain to his directorate, which includes the SAM & PM programs. Respective managers of the SAM and PM programs direct their employees toward executing and implementing those programs' activities.
2.4 Key stakeholders
Both the SAM & PM programs are performed in collaboration with the International Space Exploration Coordination Group, foreign space agencies, Government of Canada organizations and through consultations with the Canadian astronomical community. This collaborative effort takes shape under contracts, grants and contributions (G&Cs), memoranda of understanding (MOU) with other government departments, and international partnership agreements.
According to the SAM & PM programs' Performance Measurement Strategy,Footnote 7 the programs' key stakeholders include the following:
- Private enterprises (small, medium or large) involved in development of science and technology related to space exploration
- Academic institutions, research centres and universities involved in research and development of science and technology related to space exploration
- Foreign space agencies, such as the National Aeronautics and Space Administration (NASA), the European Space Agency (ESA), the Japan Aerospace Exploration Agency (JAXA), the Indian Space Research Organisation, as well as a few national space agencies in Europe
- Other government departments, such as the National Research Council Canada (NRC), the Natural Sciences and Engineering Research Council of Canada (NSERC), the Canadian Foundation for Innovation, Public Service and Procurement Canada, and the Treasury Board of Canada Secretariat. Although both NSERC and the Canadian Foundation for Innovation do not play a direct role in SAM & PM missions, they provide research infrastructure funding for Canadian scientists.
2.5 Resource allocation
The CSA's annual A-Base budget of $300 million was initially established in Budget ($215.4 million in dollars), and is now in the order of $260 million. Table 1 shows the total human and financial resources allocated to the SAM & PM programs, as well as the actual spending over the course of the evaluation period.
|Type of Resource||–||–||–||–||–|
|FTEsTable note a|
|Forecasted Budget (,000$)Table note b,Table note c|
|5-year total = 131,618 (SAM = 61,958; PM = 69,660)|
|Actual Spending (,000$)|
|Salary Table note c||2,271||2,153||1,611||987||914|
|O&M - other Table note d||892||671||743||802||904|
|O&M - contracts||4,406||5,424||4,699||3,316||1,775|
|O&M - other Table note d||154||224||65||59||53|
|O&M - contracts||2,923||2,946||1,503||1,493||1,278|
|5-year total = 110,923 (SAM = 61,258; PM = 49,665)|
Source: The CSA's Finance Directorate,
2.6 Prior evaluation of the Program
This report constitutes the first evaluation of the SAM & PM programs. The Performance Measurement Strategy for the SAM & PM programs was developed in and revised in .
3. Evaluation approach and methods
3.1 Purpose and scope
In keeping with requirements stipulated in the Policy on Results and Directive on Results, the Financial Administration Act, and the planned evaluations from the CSA's Five-Year Departmental Evaluation Plan, this cluster evaluation of the SAM & PM programs will include all aspects of the SAM & PM programs corresponding to the Program Alignment Architecture activities 184.108.40.206 and 220.127.116.11, respectively. The objective of the evaluation will be to systematically collect and analyze evidence on the relevance, effectiveness and efficiency of these programs, as well as unintended outcomes for the reporting period from to .
3.2 Evaluation issues and questions
The following evaluation issues and questions represent the key themes of the evaluation that were expanded upon and tailored to the appropriate informants in the interview protocols.
- Are the SAM & PM outcomes aligned with federal government priorities?
- Are the SAM & PM outcomes aligned with departmental strategic outcomes?
- Are the SAM & PM programs consistent with federal roles and responsibilities?
- Do the SAM & PM programs continue to address a demonstrable need?
Effectiveness – Outputs (Op)
- Have science investigations been enabled and supported? (Op1)
- Are science instruments under development or operated in space? (Op2)
- Are spacecraft or subsystems under development or operated in space? (Op3)
Effectiveness – Outcomes (Oc)
- Is there sustained access to scientific data? (Oc1 (Immediate))
- Have the number of HQP been increased? (Oc2 (Immediate))
- Has Canadian presence in space been expanded through space missions? (Oc3 (Immediate))
- Has the competence of the space sector in space exploration areas been increased? (Oc4 (Immediate))
- Is the Canadian space exploration sector better positioned to seize space opportunities? (Oc5 (Intermediate))
- Has the transfer of know-how and technology to other applications been enhanced? (Oc6 (Intermediate))
- Has knowledge about the universe, solar system and human capacity to live in space been increased? (Oc7 (Intermediate))
- Has there been sustained economic growth? (Oc8 (Ultimate))
- Has the space exploration profile in Canada and abroad been enhanced? (Oc9 (Ultimate))
- To what extent is the program delivering outputs and achieving outcomes in the most efficient manner?
- To what extent has resource use been minimized in the implementation and delivery of the program?
3.3.1 Documentation review
Documents external and internal to the SAM & PM programs were reviewed for content to evaluate the programs' continued relevance. Performance data pertaining to the achievement of the programs' outputs and outcomes over the evaluation time frame were documented in the Report on Performance Measurement for Sub-sub Programs SAM & PMFootnote 8 and a variety of files. The external documentation included various strategy papers, expert panel reports, assessments and industry submissions, as well as Government of Canada statements on the importance of research and development in the science, technology and innovation space. The internal documentation included Reports on Plans and Priorities, Departmental Performance Reports, the Performance Measurement Strategy, performance data and media tracking files, financial data files, summary reports on invitations received from international partners, and approximately 30 mission-level project files. Most of the documentation was loaded, coded and analyzed in Atlas.ti.
3.3.2 Key informant interviews
In-person group and individual interviews were conducted with CSA management and staff in Saint-Hubert. Twenty-four individuals participated in the interviews, including three director generals and the Program Director. All four planned interviews were conducted with representatives of NRC/Herzberg Institute of Astrophysics. Telephone interviews were conducted with three international partners; NASA, ESA and JAXA. All of the interviews were transcribed, loaded, coded and analyzed in Atlas.ti.
The e-survey was sent to 41 contact email addresses, one of which bounced back; 19 completed responses were received out of the remaining 40, for a response rate of 48%. The survey data cannot be considered statistically reliable because of the high margin of error. Nevertheless, valuable qualitative data were collected that provide an insight into how the SAM & PM programs are viewed by funding recipients.
3.3.4 Case studies
Two case studies were completed, one for each of the two programs under evaluation. The focus of these case studies was Canada's contribution to the NASA JWST mission and the Mars Science Lab/Curiosity rover mission. Two in-person group interviews of CSA staff involved in these missions were conducted, involving a total of six individuals. Telephone interviews were also conducted with NASA's program director for JWST, the Canadian principal investigators and an industry representative. Mission-level documentation and the transcribed interview notes were loaded, coded and analyzed in Atlas.ti.
There are some limitations to the evaluation data set that has been collected. The absence of a robust data set from international partners has limited the evaluation's ability to fully assess the effect that the programs have had on Canada's international profile and visibility in space exploration. The limited availability and quality of archival data—that is, program performance measurement records covering each of the fiscal years in the evaluation time frame—has limited the evaluation's ability to fully assess the effectiveness of output production and the efficiency of resource utilization.
The relevance of the SAM-PM programs was evaluated with regard to
- the linkages between program outcomes and federal government priorities,
- the linkages between program outcomes and departmental strategic outcomes,
- the role and responsibilities for the federal government in delivering the program, and
- the extent to which the program continues to address a demonstrable need and is responsive to the needs of Canadians.
4.1.1 Alignment with federal government priorities
Evaluation question #1 (Relevance): Are the SAM & PM outcomes aligned with federal government priorities?
Finding #1: The SAM & PM outcomes were aligned with the Canadian Space Policy Framework, prevailing federal government priorities for advancing science, technology and innovation to spur economic growth and prosperity, and recent Government of Canada announcements.
The importance of innovation as a contributing factor to future productivity and economic growth has long been accepted in Canada by government, academia and industry.Footnote 9,Footnote 10,Footnote 11 As follows, the review of federal support to research and development by a government-appointed, independent panel set out quite eloquently how innovation contributes to productivity.
In the context of productivity growth, the process of innovation diffusion and adaptation is most important, since most innovation that occurs in any given area or jurisdiction is through adaptation of signiﬁcant innovations originating elsewhere. The adoption/ adaptation by an individual enterprise of a new or better way of doing something is therefore also recognized as a form of business innovation — indeed, the most common.Footnote 12
The CSA has traditionally accounted for the largest percentage of research and development contracted to Canadian business, estimated by Statistics Canada at $167 million or 60% of the total in –.Footnote 13 The important role that the CSA has played in supporting Canadian business innovation was evidenced in the Aerospace Review. Its recommendations focused on policy and program improvement specific to the space sector, including the establishment of a Canadian Space Advisory Council and Space Program Management Board, recognition of the importance of space technologies and capacity to economic prosperity and growth, stabilized core funding, and additional funding for technology development programs.Footnote 14
Made public in , Canada's Space Policy Framework was to serve as a guide for Canada's future space programming activities, including space exploration that would inspire young Canadians to pursue studies and careers in science and engineering. The Government committed to the following points:
- Ensuring that Canada is a sought-after partner in the international space exploration missions that serve Canada's national interests
- Continuing to invest in the development of Canadian contributions in the form of advanced systems and scientific instruments as part of major international endeavours
- Continuing Canada's Astronaut Program so as to have Canadians aboard current and future space laboratories and research facilitiesFootnote 15
The SAM & PM Performance Measurement Strategy was developed and approved shortly after the Space Policy Framework was made public. It drew its guidance from the framework's first two commitments, with emphasis placed on working with international space agency partners and contributing to joint international space exploration missions. Contained within the strategy is the SAM & PM Logic Model (Appendix A), which sets out the programs' outcomes with two clear streams of logic aligned with the Government's commitments for science, technology and innovation, as outlined above. The first stream of logic is focused on enhancing Canada's space exploration profile through advancing knowledge about the universe and our solar system, developing highly qualified personnel (HQP), and providing scientific communities with access to scientific data. The second is focused on generating economic growth through the transfer of know-how and technology to terrestrial and space applications by developing the competence of the Canadian space exploration sector, and through positioning the Canadian space exploration sector for future space opportunities by maintaining a presence in space through space missions. Both streams of logic and their ultimate outcomes addressed different aspects of the Government's Science and Technology Strategy to optimize Canada's people, knowledge and entrepreneurial advantages.Footnote 16
In , the Government of Canada released its renewed Science, Technology and Innovation Strategy, the objective of which was to "
strengthen Canada's position as a global leader in scientific research and innovation"Footnote 17 and "
continue to support and deepen research across a broad spectrum of disciplines that include both discovery- and application-driven research."Footnote 18 The strategy was based upon three pillars. Encouraging young Canadians to pursue careers in science, technology, engineering and mathematics, as well as attracting and retaining HQP, were priorities of the strategy's People pillar. Supporting world-leading research with the potential to generate long-term economic benefits was a priority of the strategy's Knowledge pillar. Encouraging partnerships between academia and industry to drive innovation and position Canadian businesses in the global marketplace was a priority of the Innovation pillar.Footnote 19 This strategy affirmed the Government's intention to strengthen Canada's capacity for science and technology innovation and advanced research and development in sectors such as space exploration, where academia and industry would need government support. The SAM & PM outcomes remain well aligned with these priorities to inspire Canadians and advance space science capacity in order to position the country as a respected, innovation-driven and space-faring nation.
The Ministerial Mandate Letters for the Minister of Innovation, Science and Economic Development and the Minister of Science stated that the government intended to "
to partner closely with businesses and sectors to support their efforts to increase productivity and innovation"Footnote 20 and to ensure that "
investments in scientific research, including an appropriate balance between fundamental research to support new discoveries and the commercialization of ideas, [lead] to good jobs and sustainable economic growth."Footnote 21 The SAM-PM programs have, generally, remained well aligned with government priorities. However, the implementation of these priorities specifically as they relate to the CSA's Space Exploration Program has caused some concern among internal interviewees given recent Government announcements. It should be noted that document review and interview data indicate the current Government's priorities have been reflected in recent announcements to fund human space flight and post-ISS space exploration among competing space priorities, while the SAM-PM programs have gone without a newly approved and funded mission since the beginning of the evaluation timeframe.
4.1.2 Alignment with the CSA's priorities
Evaluation question #2 (Relevance): Are the SAM & PM outcomes aligned with departmental strategic outcomes?
Finding #2: The SAM & PM outcomes were most directly aligned with supporting innovation, providing information and generating scientific knowledge, all integral to the CSA Strategic Outcome,
The evaluation's internal document review of the CSA's Reports on Plans and Priorities and Departmental Performance Reports covering the entire evaluation time frame reveals that the CSA's Strategic Outcome has remained unchanged as follows, "
Canada's exploration of space, provision of space services, and development of its space capacity meet the nation's needs for scientific knowledge, innovation and information."Footnote 22,Footnote 23
The SAM & PM programs facilitate the participation of the Canadian space exploration sector, including academia and industry, in space astronomy and planetary missions most often sponsored by international space agency partners. These opportunities build on the existing strengths of the Canadian space industry and scientific community to contribute valued Canadian technologies and scientific expertise to the design, development and operation of spacecraft, subsystems, and/or scientific instruments.Footnote 24 Canada has demonstrated a world-class level of know-how, expertise and innovation on many international space astronomy and planetary exploration missions, making it a sought-after partner by space agencies around the world. This acknowledged capacity of the Canadian space exploration sector has taken years to develop and has given Canadian astronomers and planetary scientists access to scientific instruments and data that have led to new discoveries and advances in knowledge about the solar system and the universe.Footnote 25
The SAM & PM programs are well aligned with the CSA's Strategic Outcome and specific priorities identified in the – Report on Plans and Priorities, such as conducting "
fundamental research and new discoveries" and "
positioning of the space sector for global opportunities."Footnote 26 The interview data confirm the strong alignment of these programs with the CSA's mandate related to "
advancing the knowledge of space through science and technology."Footnote 27 The programs were acknowledged to be conducting scientific research, providing scientific data and facilitating the use of information to advance knowledge.
4.1.3 Alignment with federal roles and responsibilities
Evaluation question #3 (Relevance): Are the SAM & PM programs consistent with federal roles and responsibilities?
Finding #3: The SAM & PM programs are designed and were implemented in a manner consistent with the CSA mandate, one that is unique among federal government departments and agencies with respect to space exploration missions and technology.
The activities and objectives of the SAM & PM programs are consistent with the essential functions assigned to the CSA by the Canadian Space Agency Act, notably to "
plan, direct, manage and implement programs and projects relating to scientific or industrial space research and development and the application of space technology," to "
promote the transfer and diffusion of space technology to and throughout Canadian industry," and to "
encourage commercial exploitation of space capabilities, technology, facilities and systems."Footnote 28
In the same vein, the Aerospace Review report highlighted that
The third category of space activity is space exploration and science, which focuses primarily on satisfying our thirst and need for fundamental knowledge. The inspiring feats of astronauts, missions to the moon and other planets, space labs, and deep-space telescopes expand our understanding of the universe and our place in it. They are wellsprings of national pride and prestige, and generate technological and economic spinoffs. Such activities are almost always government-funded and, given their scale and complexity, usually carried out through international cooperation.Footnote 29
The Aerospace Review report also recommended that the government "
develop mechanisms to support the efforts of companies to keep their workforces technologically adept and adaptable through continual up-skilling."Footnote 30 Numerous other reports over the years have also encouraged the federal government to play an important role in supporting basic and applied research, as well as related training of HQP.Footnote 31,Footnote 32,Footnote 33 The National Research Council Canada (NRC) and the Natural Sciences and Engineering Research Council (NSERC), for example, have mandates related to research and the development of HQP.Footnote 34 NRC has responsibility for ground-based astronomy, but the CSA is the only agency that provides a scientific community with access to space astronomy opportunities and data. NSERC funds fundamental and applied scientific research, and was identified by e-survey respondents as another important funder of space research, although not in coordination with SAM & PM missions. Only the CSA has a mandate to "
advance the knowledge of space through science and technology" and does so in coordination with the NRC Canadian Astronomy Data Centre and with little duplication or overlap with other government departments.
The research, development and innovation that take place in the SAM & PM programs have been consistent with the CSA mandate as stated above. Providing space hardware (instruments, subsystems or whole spacecraft) requires specialized infrastructure, expertise and resources that no other government department or agency possesses. The interview data also suggest that the CSA is best positioned to develop a technologically adept workforce for the space sector, as well as HQP researchers to analyze the scientific data, in order to fulfill its scientific mandate and to contribute to Canada's competitive advantage.
Although outside the time frame of the evaluation, it is noteworthy that Budget affirmed that the federal government has an important role to play in advancing science, research and innovation in the pursuit of building a skilled and stronger middle class. Specifically for space exploration, the fact that the government decided to invest in a mission to Mars indicates that planetary exploration is a priority. Following through on this principle was an $80.9 million cash allocation to the CSA beginning in fiscal year – "
to demonstrate and utilize Canadian innovations in space" in the field of quantum technology and to support Canada's participation in NASA's next Mars Orbiter Mission.Footnote 35
4.1.4 Continued need for the Programs
Evaluation question #4 (Relevance): Do the SAM & PM programs continue to address a demonstrable need?
Finding #4: The SAM & PM programs are essential to maintaining a world-class cadre of astronomers and planetary scientists in Canada dedicated to advancing the knowledge of space through scientific discovery. In the absence of newly approved and funded missions since the beginning of the evaluation, international partners and the Canadian space industry have successfully sought out other partners and business opportunities.
Canada has a long and proud history as a space-faring nation, which is a source of national pride, prestige and inspiration for young Canadians to pursue careers in science, technology, engineering and mathematics fields. It seems only fitting that an evaluation of the continued need for these programs should be first examined from the perspective of the scientific communities involved.
The long history of astronomy in Canada, detailed in the James Webb Space Telescope (JWST) case study (Appendix B), has seen the Canadian astronomy community grow from roughly a hundred members in to approximately 300 professional astronomers and another 300 graduate and postgraduate students in universities across Canada. The community is among the most influential in the world in terms of its contribution to scientific advances in the field of astronomy and astrophysics. On the other hand, the planetary exploration community is relatively new, as described in the Mars Science Lab (MSL) case study (Appendix B). The community has grown from a small number of researchers at the turn of the millennium who conducted fundamental research that didn't require access to space, to an estimated 26 faculty members holding key research positions in top Canadian universities who have been or are currently science team members (Co-Investigators) in planetary exploration missions.
Of the SAM & PM funding recipients from academia and industry surveyed by the evaluation, 90% rated the continued need for these programs as significant, the highest rating possible. Three categories of needs were identified by the respondents:
- Advancing knowledge in astronomy and astrophysics through supporting the development of scientific instruments and technologies, enabling Canadian researchers to participate in international space missions, and producing and facilitating access to scientific data.
- Attracting highly qualified personnel and students to space-related research and improving their training.
- Supporting Canadian companies to innovate, develop and commercialize high-tech products and services.
The three main arguments put forward by e-survey and interview respondents in support of continued funding of the SAM & PM programs are summarized as follows.
- There is a need to continue the exploration of space in order to respond to fundamental questions about planets, our solar system, and the shape and composition of the universe. Respondents also indicated a need to understand the characteristics of the planet Mars and to develop new technologies to facilitate its exploration and habitation. Canadian involvement in space exploration through its leading scientists would contribute significantly to advancing knowledge in these areas.
- There are currently no funding alternatives to the SAM & PM programs to provide access to the space facilities, missions and scientific data that are essential for the conduct of world class scientific research and HQP training, without which the respective scientific communities in Canada would be diminished. It should also be noted that documented research and interview data indicate that at least 5 HQP have pursued opportunities in the United States and elsewhere, while others have also expressed their intention to do the same should there be a further decline of opportunities to conduct scientific research in their fields.
- The SAM & PM programs provide Canadian companies with the opportunity to develop advanced and reliable technologies for exceptionally challenging space applications that demonstrate their capacity to innovate, thereby raising their profiles and reputations internationally and making them more competitive in the international space marketplace.
The irregular cadence of the SAM & PM programs funding has been very challenging for the space industry leaders such as British Columbia-based MacDonald, Dettwiler and Associates (MDA) and Ontario-based COM DEV International (COM DEV), who have recently found new business partners in the United States. MDA's Canadian and US operations are now controlled by a US corporation with its headquarters in San Francisco.Footnote 36 In , the US company Honeywell announced the purchase of COM DEV International's space hardware and systems line of business,Footnote 37 leaving the space exploration business line in Cambridge, Ontario.
Similarly, international space agency partners (e.g., NASA, ESA) have formed other partnerships in the absence of Government of Canada support as evidenced in the MSL case study. The low interview response rate from international partners was revealing in this respect, as was the inability of respondents to answer many of the interview questions due to the amount of time that had passed since the last joint mission as "
these missions took place many years ago." Canadian scientists' participation in international space agency missions and subsequent access to the scientific data are generally contingent on the provision of a technology hardware contribution supplied by a Canadian space industry firm and paid for by the Government of Canada. The evaluation found that there is a strong continued need for the SAM-PM programs to maintain the world-class scientific research conducted by Canadian space astronomy and planetary exploration community stakeholders. With a committed plan of mission approvals in the future, stakeholders would continue to benefit from the SAM-PM programs.
This section presents the findings for the Effectiveness evaluation issues: output and outcome achievements. The extent to which the SAM & PM programs have achieved each of the outputs and outcomes identified in the logic model is evaluated in this section of the report, which is divided into output, immediate outcome, intermediate outcome and ultimate outcome sub-sections.
Evaluation question #5 (Effectiveness): Have science investigations been enabled and supported?
Finding #5: The number of Canadian universities and research institutes undertaking science investigations enabled through G&Cs and in operation (supported through mission-related contracts) has remained relatively steady and above the set targets over the evaluation period. Support for science investigations in development has declined markedly; the number of investigations has fallen from nine to five for SAM and from seven to one for PM, although the programs still attained their reduced targets of one (1) investigation each.
Enabling science investigations refers to the development of knowledge by the scientific community involved in space missions in astronomy or planetary sciences. Science priorities are identified through information gathered about international opportunities and through consultations with the Canadian scientific community. Proposals are solicited through announcements of opportunities and funded with grants to the university proponents. These grant-funded projects can be used to increase the science returns of archived mission data or to define the science objectives and measurement needs for identified mission opportunities. Requests for proposals with well-defined statements of work are issued on a competitive basis and funded through contracts. Typical requirements have included business case input for mission opportunities, with option analysis, costing and scheduling, and the design and development of scientific instruments or subsystem concepts as part of pre-project activities for approved missions. Science investigations are supported to perform work for the CSA related to specific requirements of upcoming or ongoing space astronomy and planetary missions.Footnote 38
Targets of two (2) and six (6) science investigations enabled by Canadian universities were set for space astronomy and planetary missions respectively over the course of the evaluation time frame. In the case of space astronomy, three universities worked on the ASTRO-H CAMS subsystem, and one other worked on the Herschel SPIRE instrument for the first four years. A fifth university began to provide support in the last year on the ASTROSAT UVIT instrument. The number of universities providing science support to the MSL APXS instrument increased from two (2) to four (4) over the course of the five years. On the other hand, the number of universities providing support to the OSIRIS-REx OLA instrument declined from four (4) to two (2) over the last three years. Two science investigations were terminated in after the withdrawal of NASA and subsequently the CSA from the ESA ExoMARS mission. Both SAM & PM exceeded their targets for grant-funded science investigations enabled over the evaluation timeframe.Footnote 39
A target of one (1) science investigation under development and supported through contracts was set for each of the space astronomy and planetary missions over the course of the evaluation time frame. The target for space astronomy was exceeded considerably given the number of missions initially under consideration (e.g., EUCLID, SPICA) and in development (e.g., JWST – NIRISS/FGS, NEOSSat, ASTROSAT – UVIT, ASTRO-H CAMS). Similarly, the target for planetary missions was only exceeded in the first two years when Mars and the ExoMARS missions were under consideration, while MSL APXS was under development. Development of the OSIRIS-REx OLA instrument was initiated in – and has remained the only science investigation under development for this program since. The number of universities, research centres and private sector companies involved declined markedly from nine (9) to five (5) for space astronomy and seven (7) to one (1) for planetary missions over the time frame of the evaluation.Footnote 40
Evaluation question #6 (Effectiveness): Are science instruments under development or operated in space?
Finding #6: Canada's science instruments on the ESA Planck and Herschel missions, as well as Canada's own MOST space telescope, were in operation until , followed by the ASTROSAT – UVIT instrument, which became operational in –. The Canadian-built APXS on NASA's MSL mission has been operational since , while the JWST's NIRISS and the OSIRIS-REx's OLA have been in development. By , SAM & PM will meet or exceed their targets with two science instruments each, delivering data on a regular basis to the Canadian scientific community.
Evaluation question #7 (Effectiveness): Are spacecraft or subsystems under development or operated in space?
Finding #7: The space telescope MOST, launched in , was the only Canadian spacecraft in operation from during the evaluation timeframe while NEOSSat, the CAMS subsystem for ASTRO-H and the FGS subsystem for the JWST were in development. ASTRO-H was launched in and declared lost in . Small space telescopes and micro-satellites were developed and launched at a reasonable cost and had very specialized scientific data gathering missions.
Canada's participation in international partner space missions is contingent on making a contribution in the form of a science instrument, subsystem or related component. Open and formal invitations to participate in international missions generally specify the scientific or technology requirements sought after by the sponsoring space agency. More personalized invitations are sometimes extended to Canada when the Canadian space sector is known for specific technologies and capabilities. For example, as described in the JWST case study, NASA sought out Canada's contribution for a mission critical subsystem in the form of an infrared tunable camera that would allow the telescope to be aimed very precisely. Based on the interview data, COM DEV's technology and demonstrated capability was key to the invitation and contract award; it also facilitated the CSA's negotiation with NASA of a scientific instrument as part of its JWST contribution, which will generate data for the benefit of Canadian astronomers and astrophysicists.
The CSA led two space astronomy missions since the inception of Canadas's space program: MOST and NEOSSat. More often, CSA has contributed science instruments and/or subsystems to other space agencies' space astronomy or planetary missions. Figure 1 presents the past, present and future operational missions by life-cycle phase over the duration of the evaluation time frame, followed by a brief description of each mission, scientific instrument and subsystem based on the available internal documentation and web research.
Figure 1: SAM & PM Missions by life-cycle phase, –
- Planck – HFI/LFI
This ESA mission was launched in , along with the Herschel Space Observatory, and ended in . The Planck mission objectives were to study the cosmic microwave background's anisotropies and polarization—exploring the birth of the universe, its evolution and the forms that it might take in the future. Planck carried two instruments: High Frequency Instrument (HFI) and Low Frequency Instrument (LFI). The CSA funded quick-look analysis and trending software and science support for both instruments, along with data reduction and post-operations support until .
- Herschel – HIFI/SPIRE
The Herschel Space Observatory mission was launched in , along with the Planck mission, and was operational and sending data until its completion in . Herschel was a European Space Agency (ESA) mission, in partnership with the CSA. Herschel was equipped with the largest space telescope ever flown and had three science instruments to analyze infrared and submillimetre radiation coming from outer regions of the universe. The three instruments were the Heterodyne Instrument for the Far Infrared (HIFI), the Spectral and Photometric Imaging Receiver (SPIRE), and the Photometric Array Camera and Spectrometer (PACS). The CSA contributed to two instruments for the mission: HIFI and SPIRE. HIFI enabled a better understanding of interstellar chemistry and detected and analyzed emissions from different molecules; SPIRE explored galaxy structure formation and studied the earliest stages of star formation.
- ASTROSAT – UVIT
ASTROSAT mission project was signed in between the CSA and the Indian Space Research Organisation. The mission is operational and as been sending data since its successful launch in . ASTROSAT is planned to be operational until . The mission objective was to study astronomical objects using ultraviolet light and X-rays. Participation in this mission means Canadian scientists are allocated observation time on the satellite, which enables them to conduct unique astronomy research. Canada's participation in ASTROSAT was funded by the CSA and the National Research Council Canada (NRC). ASTROSAT carries five instruments: Ultraviolet Imaging Telescopes (UVIT), Large Area Xenon Proportional Counters (LAXPC), Soft X-ray Telescope (SXT), Cadmium-Zinc-Telluride coded mask imager (CZTI), and Scanning Sky Monitor (SSM). The CSA provided UVIT detectors read-out electronics and calibrations. The detectors capture each photon of light and record its location and time of arrival.
- MSL – APXS
The Mars Science Lab (MSL) mission was launched in , landed on Mars in , and is still operational. MSL is led by NASA, with the objective of exploring Mars's surface with the Curiosity rover. It investigates the climate and the geology of Mars. MSL carries 13 instruments in total. Its equipment includes the following.
- Three cameras: Mast Camera (Mastcam), Mars Hand Lens Imager (MAHLI) and Mars Descent Imager (MARDI)
- Four spectrometers: Alpha Particle X-Ray Spectrometer (APXS), Chemistry & Camera (ChemCam), Chemistry &Mineralogy X-Ray Diffraction/X-Ray Fluorescence Instrument (CheMin) and Sample Analysis at Mars (SAM) Instrument Suite
- Two radiation detectors: Radiation Assessment Detector (RAD) and Dynamic Albedo of Neutrons (DAN)
The remaining two instruments are the Rover Environmental Monitoring Station (REMS) and the Mars Science Laboratory Entry Descent and Landing Instrument (MEDLI). The CSA contributed to MSL by providing the APXS instrument, which analyzes samples to determine their chemical compositions, and by supporting the research activities of three participating scientists.
- OSIRIS-REx – OLA
The OSIRIS-REx mission, carried out by NASA, was launched in . It is currently on its way to the target asteroid, Bennu. OSIRIS-REx will reach Bennu in , and spend approximately two years mapping the asteroid and searching for an appropriate sample site. The collected sample will then be returned to Earth in for subsequent investigation. OSIRIS-REx carries a total of five instruments: OSIRIS-REx Camera Suite (OCAMS), OSIRIS-REx Laser Altimeter (OLA), OSIRIS-REx Thermal Emission Spectrometer (OTES), OSIRIS-REx Visible and Infrared Spectrometer (OVIRIS), and Regolith X-ray Imaging Spectrometer (REXIS). Canada contributed to the mission by providing the OLA instrument and supporting the investigation's four additional scientists. OLA will scan Bennu's entire surface to create a 3D topographic model of the asteroid in order to set the geological context and to help select a safe sampling site.
- JWST – NIRISS
The JWST mission is currently in its development phase. Its planned launch date is -. JWST is expected to be the most powerful space observatory for at least a decade after its launch. The goals of the mission are to search and map the evolution of the earliest stars and galaxies and their formations, and search to see if there is potential for other life in the universe. JWST has four science instruments: Near-Infrared Camera (NIRCam), Near-Infrared Spectrograph (NIRSpec), Mid-Infrared Instrument (MIRI), and Near-InfraRed Imager and Slitless Spectrograph (NIRISS). The CSA contributed the NIRISS instrument, which will investigate first light detection, exoplanet detection and exoplanet transit spectroscopy.
Canadian spacecraft and subsystems:
- JWST – FGS
In addition to the NIRISS science instrument, the CSA contributed a mission-critical subsystem for the JWST. The Fine Guidance Sensor (FGS) is a tunable camera that enables the telescope to be aimed at its target very precisely. It is considered the highest precision star guidance system ever built for a space telescope application and will make it possible for JWST to obtain high-quality images.
The CSA's MOST mission was not only Canada's first Canadian-led space astronomy mission, it was the world's first astronomical space telescope on a micro-satellite. By measuring very precise oscillations in intensity of the stars it determined their ages and composition. It was also used to observe transiting exoplanets. MOST was launched in , and funded by the CSA until . After that, it was sold to a privately owned company which has continued its operation, making it available for use by science customers for a fee. This move represents the first time a Canadian space science mission was privatized.
The NEOSSat micro-satellite mission was approved in , and sponsored by the CSA and Defence Research and Development Canada. The mission development phase started in , and the satellite was launched in . The satellite has been commissioned and has faced several issues. It is now operational but does not fully meet the original requirements for the space astronomy component of its mission. The goal of the mission is to acquire useful metric data on near-Earth asteroids, assess potentially hazardous asteroids and comets. The contribution of NEOSSat to both Canada and the world is to provide information targeting on the population of near- Earth and potentially hazardous objects.
- ASTRO-H – CAMS
The ASTRO-H mission project started in . The mission was in development phase from to . ASTRO-H was launched in ; however, two months after the launch, the mission lost communications with the control base in Japan. Canada's contribution to the ASTRO-H mission began with helping towards the design of the Hard X-Ray telescope. In , JAXA asked the CSA to contribute an optical measurement system for the Hard X-Ray telescope. The CSA built the Canadian ASTRO-H Metrology System (CAMS) and delivered it to JAXA in . CAMS was designed to enable the mission operators to calibrate the data from the Hard X-Ray telescope and thus improve the quality of the images. The ASTRO-H mission was supported by the Government of Canada to ensure that Canadian astronomers continued to play a significant role in international astronomy. The mission objectives were to study the structure black holes, active galaxies, and other cosmic phenomena in extreme conditions.
4.2.2 Immediate outcomes
Evaluation question #8 (Effectiveness): Is there sustained access to scientific data?
Finding #8: Canadian instruments on space astronomy and planetary missions have transmitted a constant stream of observations and images of various types and quality over the years. Calibrated data must in principle be made accessible in public archives within six (6) months for use by all researchers. Principal Investigator/science team involvement in the instrument design and/or post-operation phases has not always been supported due to a lack of grant funding, which adversely affects data quality, data reduction and analysis, and limits the science teams' contribution to the knowledge outcome.
The – Departmental Performance Report on sub-programs and sub-sub programs states that five (5) space astronomy and planetary missions provided data to the Canadian scientific community, surpassing the set target of four (4) missions. Based on additional internal document review, as summarized in Figure 1, and the accompanying mission descriptions, the following space astronomy missions and their scientific instruments were in operation and transmitting data during the evaluation time frame: Herschel (HIFI, SPIRE), Planck (HFI, LFI), MOST and NEOSSat. On the planetary missions side, the APXS instrument has been the most consistent and productive provider of scientific data with the Canadian contribution of the APXS to the MSL Curiosity rover. These missions and their scientific instruments produced a variety of data types, including infrared, optical, X-ray and metric data, for unique research study purposes. Data quality has been an issue for NEOSSat since its launch, and testing continues to determine how the satellite can produce useful scientific data.Footnote 41
The movement towards open data has increasingly gained momentum in space science. For example, NASA requires principal investigators (PIs) to ensure that all transmitted data are calibrated and archived within six months of the download date. In principle, this has broadened public access to the scientific data and given non-mission-related scientists with the appropriate software platforms and tools the opportunity to conduct their own research.
While the closure of mission operations signals the end of the data collection phase, pre-and post-operations support for data requirements definition, instrumentation, data reduction and analysis are equally as important. The NASA program provides end-to-end funding for scientific investigations into instrument design, software development, data reduction and returned sample analysis. Based on the interview data, NASA's competitive tendering criteria require mission contribution proposals to include funding for these scientific activities. This evaluation noted that the Herschel and Planck missions ended in , that CSA support for post-operations data reduction and analysis was extended to , and that the CSA provided grant funding for the development of data analysis software for the Planck HFI/LFI instruments. However, according to the interview data, the principle of end-to-end funding for science team participation in SAM & PM missions has been unevenly applied.
The evaluation's two case studies documented the important role played by the PIs responsible for the science instruments in the success of the MSL and JWST missions to date. Close collaboration was critical with space industry partners and CSA staff to ensure that the instrument design met the data requirement parameters as defined by the science team. Early involvement of the science team in defining the mission's science requirements was identified as a key success factor in the interview data, as was an early commitment to grant funding for data analysis by the PI/science team. One of the barriers to science team participation identified in the interview data was as follows:
- The space industry partners' contract budget line item "
science team support" left ample room for interpretation as to its purpose. What is often unclear is whether the funds are intended to be used when the industry partner requires support, or to be used to support the science team in carrying out its responsibilities?
The availability of grant funding to support PI/science teams during the pre-and post-operations mission phases was considered preferable, enabling them to better guide instrumentation design, develop data analysis software/tools and support post-mission scientific research. At present, they are considered disadvantaged relative to their research colleagues, who are unencumbered by the responsibilities of operating the science instrument and meeting the open access requirements, which leave little time to contribute to the advancement of knowledge.
Evaluation question #9 (Effectiveness): Have the number of HQP been increased?
Finding #9: The number of faculty and students that have either received funding or were employed on mission-related contracts has fluctuated over the evaluation time frame given the irregular cadence of scientific investigations and missions. The SAM targets of 40 faculty and 30 students were surpassed, while the PM targets of 60 faculty and 50 students were not reached.
The CSA Annual Survey of the Canadian Space Sector has tracked the number of highly qualified personnel (HQP) by Canadian region relative to the national workforce since ; "
HQP measurement consists of tracking the number of employed engineers, scientists and technicians in the Canadian space sector."Footnote 42 Disaggregated data at the level of the sub-sub programs were not found in the available internal documentation. Issues with the reliability of the reported performance measurement data for fiscal year – also affects the reliability of the entire HQP data set, given the year-over-year fluctuation in the number of faculty members that received SAM-PM funding and the number of students and postdoc fellows who worked on SAM-PM-funded projects. Nevertheless, on average, the performance targets of 40 faculty and 30 students were surpassed at 55 and 46 respectively for the space astronomy program. As for the planetary missions program, the targets of 60 faculty and 50 students fell short at 14 and 29 respectively.Footnote 43
The evaluation's two case studies do shed some light on the fluctuation in the number of HQPs receiving CSA funding or engaged in funded projects. For the JWST mission, COM DEV had employed at least a dozen HQP throughout the project, while the universities involved approximately a dozen faculty members. During the peak periods of designing, building and testing the NIRISS/FGS flight hardware prior to delivery to NASA, the COM DEV contingent of HQPs was estimated to have peaked at 50 people, the majority of whom would otherwise not have been employed with the company. The University of Montréal will also employ additional postdoc fellows in the autumn of to analyze the NIRISS data, developing expertise that would not otherwise be possible.
The MSL APXS case study followed a similar pattern, with a small core group of HQP working for MDA for the duration of the project, along with 10–15 university faculty and students working with the Principal Investigator. The number of HQPs employed by MDA may not have peaked to the same extent as at COM DEV due to the established nature of the APXS technology; however, the University of Guelph will employ additional faculty and students to continue to analyze the wealth of data that it has collected.
The very significant fluctuations in HQP performance measurement data at this sub-sub program level may be more understandable given the observed swings in HQP employment reported by the space industry and university partners over the course of a project's life cycle.
Evaluation question #10 (Effectiveness): Has Canadian presence in space been expanded through space missions?
Finding #10: There have been at least two space astronomy missions and one planetary mission in operation in any given year over the evaluation time frame. The importance of NASA's MSL and JWST missions, with the Canadian APXS and NIRISS/FGS science instruments and subsystems playing important roles, has at least enhanced Canada's reputation if not expanded its presence in space exploration.
As illustrated in Figure 1 above, at the beginning of the evaluation time frame Canada had three (3) space astronomy missions (Herschel, Planck, MOST) and one (1) planetary mission (MSL) were in operation. The three space astronomy missions ended operations in . Canada's space astronomy missions experienced some setbacks with the NEOSSat data quality issues faced since its launch in , and the unfortunate loss of JAXA's ASTRO-H mission with its Canadian payload in . The Indian ASTROSAT mission, with the Canadian UVIT science instrument, is operational and has been sending data since its successful launch in ; it is currently the only operational space astronomy mission that gives Canada a presence in space.
NASA's MSL mission, with its Canadian APXS science instrument, has been operational since and is the only planetary mission that currently gives Canada a presence in space. It is a high-profile and by all accounts successful mission, to which Canada has made a significant contribution as detailed in the case study. It should also be noted, that NASA's OSIRIS-REx mission with the Canadian OLA science instrument aboard was launched in and will reach the Bennu asteroid in , while NASA's high-profile JWST mission with its Canadian payload (i.e., NIRISS/FGS) is scheduled to launch in - and will become operational one year later. The evaluation found that the important contribution made to this prestigious scientific undertaking has enhanced Canada's reputation. In sum, Canada's role and contributions in space astronomy and planetary missions over the evaluation time frame have enhanced Canada's presence in space.
Evaluation question #11 (Effectiveness): Has the competence of the space sector in space exploration areas been increased?
Finding #11: The competence of university faculty researchers and students involved in scientific investigations related to space astronomy and planetary missions has improved and even attracted support for the creation of specialized research institutes. However, the irregular cadence of these programs has not benefited the space industry partners, who are challenged to maintain revenue streams and tend to lose valuable HQP between contracts.
The interview and e-survey data indicate that the SAM & PM programs have positively affected the competencies of academia and space industry partners active in the space exploration sector. Based on their personal experiences as members of a mission science team, new and enhanced competencies were achieved in software development, instrumentation, image acquisition, and modulation and data management, among others. From an institutional perspective, participation in space missions facilitated the recruitment to science programs of postdoctoral fellows and high-performing students, whose learning accelerated as a result of working on mission-related projects. Furthermore, the possibility of participating in highly visible space missions and/or research has spurred both young scientists' interest in space exploration and institutional development, as evidenced by the constitution of research teams at the University of Guelph and the creation of the Institut de Recherche sur les Exoplanètes at the University of Montréal, respectively.
The space industry partners also acknowledged the enhanced competencies accrued through their involvement in space astronomy and planetary missions, specifically in terms of large-scale project management, technology innovation, collaboration with academia and international space sector relations experience. These competencies could be reinvested in future space missions and commercial applications. Of the e-survey respondents, 48% assessed the improvements to their competencies as equal to or higher than what was expected, 38% responded that this was to a considerable extent, and 11% to some extent. Key competency areas identified by both stakeholder groups included science instrument development, software development and techniques for data processing, business development, and proposal preparation.
It is noteworthy to point out that space industry partners, even though improvement in competencies is reported, found it challenging to maintain their revenue streams and retain HQP between mission contracts. The sporadic and irregular cadence of SAM & PM mission approvals, with none over the past five years, has been a cause of concern because capacities and competencies dwindle in the absence of new contract opportunities. The procurement rules and practices exercised by foreign space agencies, who also favour their national space industries and scientists when awarding contracts, makes the successful development of alternate space markets challenging.
4.2.3 Intermediate outcomes
Evaluation question #12 (Effectiveness): Is the Canadian space exploration sector better positioned to seize space opportunities?
Finding #12: Canadian university researchers and space industry companies are frequently sought after for their expertise, technology and can-do attitude by international space agencies and other partners. Their demonstrated capability in successfully delivering critical mission components and scientific instruments positioned them for follow-on missions, although some, such as Mars and New Frontiers-4, could not be supported.
The Canadian space exploration sector's involvement in space astronomy and planetary missions was relatively modest given the total number of Canadian universities and space sector companies that could have potentially been involved. Internal document review revealed that there were 14 Canadian universities participating in national and international space astronomy missions in and the same amount at the end of the evaluation time frame in ; for planetary missions, the number of Canadian universities involved rose from 9 to 12 during the same time frame. Four (4) Canadian space sector companies participated in space astronomy missions and two (2) participated in planetary missions over the course of the evaluation time frame.
Although not large, this active segment of the space exploration sector was sought out over the years for its technological expertise and scientific knowledge in astronomy and planetary sciences, such as the design of optical systems for cryogenic applications, star tracking algorithm expertise, science algorithm expertise, data management and processing, and robotics. A compilation of invitations to participate in international missions from NASA, ESA, JAXA, and the Russian and Indian space agencies attest to the high regard with which Canadian academia and industry partners are held in the international community. The interview data collected in the context of the JWST and MSL case studies confirm that the high visibility of these two NASA missions contributed considerably to Canada's international competitiveness by demonstrating the application of innovative technologies, engineering and scientific know-how. Successful collaboration between academia, industry and government in both cases was noted as exceptional and uncommon in other jurisdictions.
Past experience, demonstrated expertise and a successful track record of accomplishments have led to an increasing number of space exploration opportunities. For example, the interview data attributed the involvement of Canadian scientists in the ESA Planck mission as key to their participation in the ESA Euclid mission; the Canadian contribution to NASA's Far Ultraviolet Spectroscopic Explorer (FUSE) mission, with fine error sensors used to stabilize and point the telescope, was attributed with the invitation from NASA to contribute a Fine Guidance Sensor for the JWST; MDA's role in building the Mars Phoenix lander was attributed with a successful proposal for an ESA ExoMars rover contract; and, more recently, Canada's contribution to NASA's Phoenix lander and MSL Curiosity rover missions was attributed with a competitive and successful bid to participate in the OSIRIS-REx mission.
The e-survey response data confirm these findings, as 42% of the respondents met or exceeded their expectations to undertake follow-on space exploration work, while 37% met their expectations to some extent. Several examples were cited of direct attribution: having Canadian scientists involved in the ESA Herschel mission was attributed with an invitation to participate in the JAXA Space Infrared Telescope for Cosmology and Astrophysics (SPICA) mission scheduled to launch in –; similarly, the development of the Canadian ASTRO-H Metrology System (CAMS) for the JAXA ASTRO-H mission was attributed with a successful contract bid by the company to provide the same laser metrology system technology for ESA's Proba-3 dual satellite mission.
There were, however, some opportunities for the Canadian space exploration sector that were not seized for which Canada was well positioned; these include NASA's Mars and New Frontiers-4 missions, the circumstances of which are described in detail in the evaluation case studies. Other opportunities were not pursued as well, as evidenced by the lists of invitations the CSA received for space astronomy and planetary missions over the evaluation time frame.
Evaluation question #13 (Effectiveness): Has the transfer of know-how and technology to other applications been enhanced?
Finding #13: The know-how acquired from being a Principal Investigator, science team member or engineer on a space mission has facilitated the mobility of HQP within the space sector across academia, industry or government. The transfer of space technologies to terrestrial settings occurred in some cases related to optics, robotics, fuel cell and laser technologies, whereas the potential reuse of space technologies for other space missions is clearly a competitive advantage and occurred more often.
The evaluation has already assessed in this report the extent to which competencies were enhanced for both industry and academic partners through the participation in space astronomy and planetary missions. The knowledge, expertise and confidence that were acquired with past mission experience, as well as the innovative technologies that were developed, could have been transferred to other space-related or terrestrial settings. The extent to which this occurred has not been rigorously studied, documented or reported. The notable exception was found in the – Departmental Performance Report,Footnote 44 which cited seven examples of space exploration knowledge and technology reuse; Opal-360, LiDAR, Nuvu Cameras, Sitelle, Xiphos Technologies, NORCAT, and Key Mars Rover Elements.
The flux of large and small space astronomy and planetary mission projects involving Canadian industry and academia over the past decade has created a unique space exploration sector demanding a high degree of mobility among HQP. As previously mentioned, the attraction and retention of HQP by the space industry firms involved has been challenging; according to the interview data, the acquired know-how is often transferred to another setting in academia, government research labs or other recently successful space industry companies. The acquired know-how is never lost to the extent that HQP tend to remain within the Canadian space sector; however, that has not always been the case due to the lack of opportunities, as previously described.
The e-survey data are telling in this regard, as 31% of respondents either met or exceeded their expectations for know-how and technology transfer to other space exploration applications, and only 10% met or exceeded their expectations for transfer to terrestrial applications. The continued need for a robust space exploration program was a critical factor in the extent to which the acquired know-how is transferred.
The technologies developed for space as described in the JWST and MSL case studies were designed to operate in cryogenic and otherwise extreme environments for which there are few, if any, parallels on Earth. Highly innovative and proven to meet exacting performance standards, these technologies may only find a terrestrial application in the distant future as the innovation process is not predictable, linear or fast. The interview data provided some "
spin-in" and "
spin-out" examples, such as MDA's light detection and ranging (LiDAR) technology. At the very least, these companies can claim to have a technology innovation edge, having proven performance under the most extreme conditions.
A few technologies developed for space uses that were then "
spun-out" for terrestrial applications were also identified in the interview and e-survey data; they included a terahertz microscope for breast cancer diagnosis, a spectrometer for nuclear fusion diagnostics, miniature digital cameras for biomedical usage, Mars rover platforms used in heavy duty vehicles, remote sensing instruments for extreme environments, and aerial observation technologies. Some of these examples overlap with those previously cited in the – Departmental Performance Report and appear to be common points of reference to evidence the potential for economic benefits. What is more evident from the previous finding and supporting text, however, is the high potential for the reuse of know-how and innovative technologies in future space missions.
Evaluation question #14 (Effectiveness): Has knowledge about the universe, solar system and human capacity to live in space been increased?
Finding #14: With a long history of access to ground and more recently space observatory data, Canadian astronomers have been recognized as leaders for the quality and impact of their research. The near doubling of scientific production of SAM-funded researchers appears to bear this out, although PM-funded researchers have not fared as well.
The JWST case study sets out a brief history of astronomy in Canada, the expansion of the Canadian astronomy community, the development of ground-based and space observatories, and the growth of its professional association, the Canadian Astronomical Society, which plays an influential role in setting the scientific priorities for the community. While somewhat dated and broad in scope, a bibliometric study of the performance and impact of Canadian research in astronomy and astrophysics drew the following conclusion based on the industry standard indicators of average of relative citations (ARC) and the average of relative impact factors (ARIF):
Canada stands out among world leaders in the field, mainly for the impact of its research on the scientific community (second place for ARC) and for the quality of its research (first place for ARIF).Footnote 45
More recent bibliometric data collected by the CSA library indicate a continued growth in scientific production by SAM & PM funded researchers, as measured by the number of peer-reviewed scientific publications produced. The number of publications almost doubled in the case of space astronomy, from 82 to 149, while that of planetary sciences increased from 15 to 20 over the evaluation time frame.Footnote 46 The interview data also confirm the scientific production of the space astronomy scientists involved in the ESA Planck mission.
In exchange for Canada's support, Canadian researchers were able to obtain access to the data collected by the mission on the cosmic microwave background. An early data release took place in , followed by a public release of data in and a further, more complete release in . 28 research papers relating to the data were published in Astronomy & Astrophysics in .
Similarly, the interview data with regard to the ESA Herschel mission indicated that the HIFI science instrument produced a strong scientific output about detecting the presence of water in space, and resulted in 33 published papers, 25 of which were directly linked to the guaranteed time that Canada received. On the planetary mission side, the APXS data was also widely used by the science team, two thirds of whom used the data in over 60 peer-reviewed publications. Although outside the time frame of the evaluation, the Canadian NIRISS science instrument onboard the JWST also has a very high potential for important scientific discoveries as a result of the 420 hours of allocated observation time.
4.2.4 Ultimate outcomes
Evaluation question #15 (Effectiveness): Has there been sustained economic growth?
Finding #15: The evaluation found little evidence that the SAM & PM programs can be attributed with sustained economic growth, although a handful of cases were identified where spin-in and spin-offs of space technologies appear to have some commercial and economic potential. Also, the fact that the two major industry partners (MDA, COM DEV) merged with or sold key assets to US companies in is not a favourable indicator of future economic growth.
Sustained economic growth is generally understood to be the increase in the value of goods and services produced by an economy for an extended period of time without interruption. Based on the financial statements provided by the CSA's Finance Directorate on , previously presented in this report (Table 1), the SAM & PM programs disbursed $111 million over the five-year evaluation time frame. The flow-through to the space exploration sector—based on the breakdown by disbursement category of grants and contributions, capital expenditures and contracts—amounted to 83.5% of the total, or $92.7 million. By comparison, the Canadian space sector generated total revenues of $3.4 billion in Footnote 47 and $5.3 billion in .Footnote 48
A closer analysis of space revenues by subsector of activity permits a general assessment of the relative importance of the SAM & PM programs to economic growth. Revenues for the space exploration sector were estimated at $114 million in ; they decreased to $86 million by ,Footnote 49 before rebounding to $112 million in .Footnote 50 Space exploration revenues declined relative to the space sector, from 3.35% to 2.11% of total revenues between and . With combined total flow-through disbursements of $92.7 million, or $18.54 million annually, to the space exploration sector stakeholders, the SAM & PM programs represented approximately 16.5% of the space exploration subsector's revenues and less than half of a percentage point of the total revenues of the Canadian space sector as a whole. While the direct economic stimulus may not be significant, it may be worthwhile to examine the indirect economic benefits of these programs.
The evaluation examined the extent to which the SAM & PM projects led to follow-on funding by other organizations and/or contracts with other space agencies. The e-survey data indicated that there were few alternatives for space exploration research projects. Of the survey respondents, 63% identified these programs as a sole source of funding, while 37% reported other sources—notably research granting councils (e.g., NSERC) and researchers' own organizations. The SAM & PM funding contribution was estimated as more than 70% of the total project cost for 80% of the respondents. A singular and significant exception was identified in the JWST case study, where the University of Montréal has been endowed with $2 million over five years from the Trottier Family Foundation to establish a new Institute for Research on Exoplanets. It will fund students and postdoctoral fellowships to analyze the data collected by the NIRISS science instrument and publish the research findings.
Follow-on contracts with other space agencies are uncommon for individual scientists and the space industry stakeholders, although a number of cases have been identified in this report. The interview data identified the following challenges facing the Canadian space industry's attempts to export their space-related goods and services: procurement policies and practices that favour the use of national scientists and firms, export control laws and regulations (such as the US International Traffic in Arms Regulations), as well as strong international competition. A perceived strength of the Canadian space industry is to leverage innovations developed through participation in space missions and use them to establish niche areas of global technology leadership in optics, robotics, fuel cell and laser technologies.
The Canadian light detection and ranging (LiDAR) technology was one of the most frequently cited examples of a spin-in technology and space-use application cited in the interview data, and this evidence was supplemented by web-based grey literature. LiDAR uses light in the form of a pulsed laser to measure variable distances and was extensively used in Canada for terrestrial applications such as remote sensing. It was further developed for space applications by companies such as Toronto-based Optech and MacDonald, Dettwiler and Associates (MDA), who built the weather sensing system as Canada's contribution to NASA's Phoenix Mars lander. MDA then applied this expertise and technology to Canada's science instrument on NASA's OSIRIS-REx mission, launched in . The OSIRIS-REx Laser Altimeter (OLA) assists in navigating the spacecraft towards the Bennu asteroid; it will scan the asteroid, create a 3D digital map and assist in landing the spacecraft. The Ottawa-based Neptec Design Group also has a long history of using laser technology for space applications in the US Space Shuttle program and the International Space Station. Its TriDAR (i.e., Triangulation and LiDAR Automated Rendezvous and Docking Sensor) technology, which is used to deliver cargo to the space station, has considerable potential in the evolving commercial space market. Further development of advanced space-based sensors and autonomous systems for space navigation led to a European export contract to develop and deliver navigation and localization cameras for the ESA ExoMars mission.
One of the most frequently cited examples of a spin-out technology and terrestrial application cited in the interview data involved the Lethbridge Alberta-based company Blue Sky Spectroscopy. A review of grey literature obtained by the evaluation revealed that the company "
leveraged more than two fold through contracts from the European Space Agency, International partners and public-sector collaborators to develop a Terahertz microscope for breast cancer diagnosis." CSA funding to support the Canadian Principal Investigator on ESA's Herschel mission to design and develop the Spectral and Photometric Imaging Receiver (SPIRE) instrument contributed to the development of the technology and its medical application. The technology will allow pathologists to pre-screen biopsy samples to first determine if they have a cancerous signature before further diagnostic processing, thus reducing the time and cost associated with cancer detection.
Assessing the contribution to sustained economic growth of these or other technology innovations applied to terrestrial and space settings would require extensive research. It would also involve the collaboration of the space industries directly involved and sufficient evidence to attribute the economic benefits in terms of increased revenues and employment to SAM & PM program funding for design, build and launch contracts and/or space research grants and contributions. While such an endeavour is beyond the scope and resources available to this evaluation, it is a necessary undertaking to demonstrate outcome achievement. Nevertheless, the examples cited herein have considerable potential to contribute to economic growth—assuming that the companies involved remain financially viable and are not merged with or sold to foreign corporations and their assets moved outside of Canada.
Two instances of this latter scenario occurring are worth noting based on the case study data and previously reported herein. MDA's Canadian and US operations are now controlled by a US corporation.Footnote 51 In , the American multinational conglomerate Honeywell International Inc. announced the purchase of COM DEV International's space hardware and systems line of businessFootnote 52 and merged it with the company's defense and space business line; it did not include in the purchase COM DEV's space exploration business line in Cambridge, Ontario. These events may impact sustained economic growth in a Canadian space exploration sector that has not grown over the past five years.
Evaluation question #16 (Effectiveness): Has the space exploration profile in Canada and abroad been enhanced?
Finding #16: The SAM & PM programs have enhanced Canada's space exploration profile and visibility with key international partners for the quality of the contributions made to mission subsystems and science instruments. Canada's space exploration visibility abroad may be reduced without participating in proposed missions.
CSA staff and Canadian science teams and space industries have played important roles in NASA's high-profile JWST and MSL missions, while the APXS and NIRISS/FGS instruments and subsystems have fulfilled and will fulfil critical scientific functions, as described in the evaluation's two case studies. Canada's reputation with NASA as a trusted and reliable partner in space exploration has been enhanced because of these contributions. The Canadian space sector has long had a can-do reputation among international space agency partners for delivering high-value science instruments and subsystems with low budgets and on schedule. The interview data also confirm that Canadian scientists are highly regarded by their international peers and are sought after teammates. The overall positive sentiment expressed in the interview data was also reflected in the e-survey data, as 79% of the respondents assessed the impact of the SAM & PM programs on Canada's space exploration profile and visibility as equal or superior to their expectations, citing publications, press conferences, press releases, speaking engagements and feedback from international space agency personnel.
Canada's responsiveness to invitations to participate in space missions has not always been clear or timely. For example, for NASA's Mars , the Government of Canada confirmed that it would not participate only near the end of the instrument selection process. This came as a disappointment to international partners and Canadian stakeholders who had submitted proposals at risk, without formal support from the CSA.The interview data that could be collected from this important stakeholder group were telling in this regard: "
our cooperation with Canada in space science has been spotty at best in the recent decade," and "
Canada once played an important role, so it would be regrettable if the capacity and capabilities developed in Canada were lost." The profile and visibility of the SAM & PM programs among Canadian and international partners is perceived to be at risk: "
There is a strong wish, and hope, that Canada should be more involved."
With media coverage and interest, 267 national lay press articles were published about Canadian space astronomy missions or missions with Canadian participation, whereas 460 international lay press articles were published over the evaluation timeframe. Similarly, 245 national lay press articles were published regarding Canadian participation in planetary missions, whereas 376 international lay press articles were published.
This section presents the findings for the efficiency evaluation issues: implementation process efficiencies, leveraging, adequacy of resources and cost-efficiencies.
Evaluation question #17 (Efficiency): To what extent is the program delivering outputs and achieving outcomes in the most efficient manner?
Finding #17: The declining number of SAM & PM missions in development and in operation over the evaluation time frame is commensurate with the decline in new investments, and some unfortunate mission setbacks over which the programs had no control. The sustained data access and increased competence outcomes for the Canadian scientific community have been achieved at a very reasonable cost to the Government of Canada in comparison to the overall mission costs of its international partners.
Perceptions of efficiency in the interview data varied widely depending on the key stakeholder group. Interview respondents were asked to rate the two programs separately on a five-point scale. The internal interview respondents' average efficiency rating for SAM & PM was 4.5 and 3.8 respectively, while external interview respondents gave ratings that were 3 points lower for the planetary missions. From the perspective of the SAM & PM funding recipients, 26% rated the impact of these programs on the development of new technologies, instruments and the generation of new knowledge as more than that of equivalent programs, while 42% rated the performance as below that of equivalent programs. Some of the principal explanations drawn from the interview data for these efficiency ratings are summarized below.
The absence of a long-term strategic plan was attributed in the internal and external interview data with the adoption of an ad hoc and reactive decision-making process in response to the priorities identified by the Canadian science community and invitations received and in mission approvals; this constrained the capacity of Canada to develop an innovative research program in space exploration and to prioritize its missions accordingly. The uncertainty of funding and the lengthy decision-making process required for mission approvals imposed constraints on the CSA's capacity to commit to proposed space missions and adapt to the time frame of its space agency partners. This incurred significant costs on academia and industry partners positioning themselves for upcoming missions, collaborating on proposals and advocating for their approval. Also worth mentioning, there are instances where the CSA has committed to a mission, such as the case of MATMOS, but international partners withdrew, and consequently Canada's contribution was not required anymore. And, finally, inefficiencies in resource utilization for the CSA, industry and academia were perceived to have increased as a larger portion of technology upgrades, mission contribution studies, phase 0 studies and proposals were unsuccessful in obtaining mission approvals, largely due to funding constraints to engage in missions.
In contrast, the higher efficiency ratings given by internal interview respondents were justified because of the leveraging effect, which augmented cost-effectiveness. With relatively modest investments, the SAM & PM programs were successful in obtaining access to space mission infrastructure that Canada on its own could never afford. The evaluation's two case studies present the evidence in this regard in considerable detail and are summarized below for convenience.
- NASA's JWST mission has cost over US$9B with development delays, while the Government of Canada's contribution for NIRISS/FGS is estimated at CAD$214M.
- NASA's MSL mission has cost over US$2.5B with extensions in mission operations, while the Government of Canada's contribution for APXS is estimated at CAD$20M.
In exchange for these relatively modest investments come numerous benefits: scientific, cultural and even economic. The Canadian astronomy and planetary sciences communities have opportunities to influence the direction of the space research, design and deliver their own scientific instruments, have guaranteed time of observation (when applicable), and priority access to the research data for at least six months. These advantages are not negligible and serve to maintain Canada's influential position among the leaders in space exploration research, which is a source of pride and inspiration to attract and retain highly qualified personnel in the science, technology, engineering and mathematics disciplines that are essential to an innovative and productive Canadian economy.
Evaluation question #18 (Efficiency): To what extent has resource use been minimized in the implementation and delivery of the program?
Finding #18: Both human and financial resource use have been reduced to their near lowest points in fiscal year –, reflecting the significantly lower levels of program activity and disbursements. The combine overhead rate for –, at 20%, was high relative to other Government of Canada programs with grant and contribution components (12%–17%).
The evaluation undertook a budget versus actual disbursements variance analysis based on the financial data provided by the CSA's Finance Directorate, presented in Table 1. Year-over-year variances for SAM & PM are presented in figures 2 and 3 below. While the variances themselves are unremarkable, the downward trend in budget estimates and actual disbursements for both programs since fiscal year – is noteworthy. Both programs had their near lowest disbursements at $7.3 million in fiscal year –, and lowest staffing levels at 9.5 FTEs and 6 FTEs for SAM & PM respectively. Based on the interview data, several staff members had already been transferred to other areas of the Space Exploration Program. The decline in disbursements over the evaluation time frame can be attributed to the maturity of four ongoing missions that were either in their operational or data reduction life-cycle phases and the lower costs associated with the development phases of four smaller satellite missions. While the JWST had incurred significant costs during the development phase of the NIRISS/FGS, these costs declined significantly after the delivery of the Canadian payload in . The last factor of note to explain the decline in disbursements was the fact that there were no approved missions in the concept and design phase over the course of the evaluation time frame as illustrated in Figure 2.
Figure 2: SAM Planned vs Actual disbursements in $1,000s, –
Source: Financial data provided by the CSA's Finance Directorate
Figure 3: PM Planned vs Actual disbursements in $1,000s, –
Source: Financial data provided by the CSA's Finance Directorate
The structure of SAM & PM disbursements illustrated in Figure 4 reveals that Capital and O&M – Contracts were the largest categories for both programs, although in different proportions, reflecting the flow-through nature of the disbursements to universities and space industry partners.
Figure 4 : Distribution of total disbursements in $1,000s, –
Figure 4: Distribution of total disbursements in $1,000s, – - Text version
Histogram, a graph depicted by bars, displaying the distribution of total disbursements in $1,000s from to . There are 2 bars divided in 5 sections (G&C, Capital, O&M-contract, O&M-other, Salaries).
|Space Astronomy Missions: (Sub-Sub-Program 18.104.22.168)||Planetary Missions: (Sub-Sub-Program 22.214.171.124)|
|O&M - contract||$19,620||$10,142|
|O&M - other||$4,012||$556|
Source: Financial data provided by the CSA's Finance Directorate
The annual overhead rates were calculated for SAM & PM and are presented separately in tables 2 and 3 and combined in Table 4. SAM had the second highest single annual overhead rate (25%) and highest average overhead rate (20%). Despite very significant reductions in staffing from 17 to 6 FTEs and the concomitant drop in salary burden, the more significant decline in actual flow-through costs caused the overhead rate to increase.
|Disbursement Type/ Fiscal Year||–||–||–||–||–|
|Actual salaries $||2,271,000||2,153,000||1,611,000||987,000||914,000|
|Actual O&M – other $||892,000||671,000||743,000||802,000||904,000|
|Actual flow-through costs $||13,595,000||14,611,000||11,761,000||13,942,000||7,348,000|
Source: Financial data provided by the CSA's Finance Directorate
PM had the single highest annual overhead rate (29%) and lowest average overhead rate (16%). Staffing did not decline significantly over the period, moving from 11.2 to 9.6 FTEs, nor did the salary burden. PM's "
Actual O&M – other" costs were considerably less than those of SAM, which along with a peak in actual flow-through costs in – would account for the much lower overhead rates.
|Disbursement Type/ Fiscal Year||–||–||–||–||–|
|Actual salaries $||1,222,000||1,091,000||1,173,000||1,120,000||1,117,000|
|Actual O&M – other $||154,000||224,000||65,000||59,000||53,000|
|Actual flow-through costs $||4,771,000||8,486,000||18,777,000||10,290,000||7,342,000|
Source: Financial data provided by the CSA's Finance Directorate
In Table 4 we can clearly see the overall decline in actual flow-through costs beginning in fiscal year –, and although staff reductions followed this date, the combined overhead rate at the end of the period was 20%. This is higher compared to the 12%–17% overhead rates of other Government of Canada programs with grant and contribution components.
|Disbursement Type/ Fiscal Year||–||–||–||–||–|
|Actual salaries $||3,493,000||3,244,000||2,784,000||2,107,000||2,031,000|
|Actual O&M – other $||1,046,000||895,000||809,000||862,000||957,000|
|Actual flow-through costs $||18,366,000||23,460,000||30,555,000||24,363,000||14,728,000|
Source: Financial data provided by the CSA's Finance Directorate
Despite staff and cost reductions, increases in the overhead rate is an indication of a program in decline, as actual flow-through costs decrease in the absence of new funding approvals. The current level of SAM & PM program flow-through costs to the academic and industry partners in space exploration sector is so small ($14.7M in -) relative to overall space exploration revenues that it could not serve as a stimulus for sustained economic growth, other than at an individual enterprise level.
5. Conclusions and recommendation
This section presents the evaluation conclusions based on the findings and supporting evidence, as well as the proposed recommendation to address the key issues identified in the conclusions.
Conclusion #1: The SAM & PM programs are well aligned with the government's innovation agenda and the CSA's mandate and strategic outcome. The SAM & PM programs are essential to maintaining a world-class cadre of astronomers and planetary scientists in Canada dedicated to advancing the knowledge of space through scientific discovery. They have not, however, received the funding approvals to pursue new opportunities during the five-year evaluation period. In a context of constrained resources, these programs may encounter difficulty in addressing the long-term needs of Canadian astronomers and planetary scientists, as well as space industry and international partners. [Findings: 1, 2, 3, 4]
The evaluation has found that the SAM & PM programs are highly relevant to the CSA mandate and its strategic outcome by developing Canada's space exploration capacities to advance the knowledge of space through science and technological innovation. The current space astronomy and planetary missions that are in the development and operational phases will continue to meet the needs of their respective scientific communities with opportunities to conduct space exploration research, access scientific data and contribute to the advancement of knowledge for several years. However, there were no missions in the concept and design phase during the evaluation time frame that would eventually replace the current set of missions within the 5- to 10-year time frame. As the CSA is the only agency that provides a scientific community with access to space astronomy opportunities and data, and without a mission approval in the very near future, the SAM & PM programs are therefore at risk of not meeting the needs of the scientific communities and academia partners in the near future.
5.2 Effectiveness and Efficiency
Conclusion #2: The number of science investigations enabled, in development and/or in operation met lowered target expectations relative to the eight (8) missions in operation targeted for fiscal year –. This decline was due to mission setbacks and the non-approval of proposed missions requiring Government of Canada financial commitments in advance. Canada's participation in international partner space missions is contingent on making a contribution in the form of a science instrument, subsystem or related component. The space astronomy and planetary missions that Canada has led (i.e., MOST) or—more often—contributed to included one or more science instruments and/or subsystems. By , SAM & PM will meet or exceed their targets with two science instruments each, delivering data on a regular basis to the Canadian scientific community. [Findings: 5, 6, 7]
The performance expectations of the SAM & PM programs at the beginning of the evaluation time frame in were reflected in the CSA Report on Plans and Priorities – and reported against in the Departmental Performance Report –. It was expected at that time that there would be eight (8) CSA-sponsored space astronomy and planetary missions in operation and providing data for the Canadian scientific community. While the target was partially met in that fiscal year, the evaluation findings illustrated in Figure 1 show that the number of missions in operation and transmitting data had diminished with the closure of several missions (i.e., Herschel, Planck, MOST), the loss of the ASTRO-H mission, the data quality issues of the NEOSSat mission and the launch delay for the JWST mission. By fiscal year – there were two operational missions with Canadian payloads (MSL – APXS, ASTROSAT – UVIT) and no additional missions approved and in the concept and design phase. The CSA organizational context within which the SAM & PM programs had been operating had become increasingly constrained by the lack of financial resources due to the JWST mission and other CSA priorities, thus limiting management's flexibility to approve additional missions.
Conclusion #3: Canadian instruments on space astronomy and planetary missions have transmitted a constant stream of observations and images of various types and quality over the years. However, at times, the engagement of the science teams was delayed due to a lack of grant funding at start-up and fragmented funding for scientific research during and after missions. [Findings: 8]
Despite the emphasis and focus placed on the CSA mandate to advance the knowledge about the solar system and the universe through space astronomy and planetary missions, the evaluation found that the principle of end-to-end funding for science team participation in space missions, which is generally accepted as a standard best practice, was evenly applied. Comments from key informants generally revolved around the presence or absence of timely and appropriate funding mechanisms.
Recommendation #1: The program should develop clear guidelines that clarify the definition and scope of science support to missions in order to allow for the continuity of science support funding throughout all phases of a mission from pre-definition phase to post-operation activities
Conclusion #4: The achievement of significant outcomes were noted in the JWST and MSL missions, where government, academia and space industry partners worked collaboratively to deliver high-performance scientific instruments (NIRISS, APXS) and critical subsystems (FGS). As a result, the Canadian space exploration sector is better positioned to seize space opportunities, and reuse its know-how and technology in future space missions to generate scientific discoveries. [Findings: 12, 13]
Space exploration research is a high-risk undertaking with uncertain returns even when successful. The loss of the ASTRO-H spacecraft, the NEOSSat scientific data quality issues, and unexpected mission cancellations and delays has associated performance and financial costs. Nevertheless, the assessment of outcomes from a quantitative perspective were in keeping with the number and size of the SAM & PM missions, while the assessment of outcomes from a qualitative perspective were impressive. The two mission case studies demonstrated a high degree of collaboration among the Canadian government, space industry and academic partners, and collectively with their international space agency partners. The transfer of know-how and technology from one mission to another has enhanced the competencies and capacity of this segment of the Canadian space exploration sector to position itself for future space opportunities.
Conclusion #5: The sustained data access and increased competence outcomes for the Canadian scientific community have been achieved at a very reasonable cost to the Government of Canada in comparison to the overall mission costs of its international partners. In addition, Canada's international profile and reputation have been enhanced with each successfully completed mission deliverable, particularly those for the high-profile JWST and MSL missions. However, the fact that there had been no missions approved during the evaluation period has produced some perceptible consequences among both Canadian and international partners, which could adversely affect Canada's reputation as a preferred partner in space astronomy and planetary exploration. [Findings: 11, 15, 16, 17]
The evaluation findings are unequivocal with respect to the positive contribution that the SAM & PM missions have made to Canada's space exploration profile and reputation. The combination of space-tested technologies, scientific and engineering know-how, and can-do attitude are often-noted characteristics. Recognized as a trusted and reliable NASA partner, the CSA and its partners have been able to forge new partnerships to undertake space astronomy and planetary missions with the increasingly active space agencies of several other countries, including Japan and India and to export tested technologies and scientific instruments to other countries, such as, Poland and Austria.
Conclusion #6: During the evaluation period, the SAM & PM programs considerably reduced the amount of capacity in terms of staffing, operations and maintenance budgets, amongst others, in an effort to adjust to the decline in mission activities and concomitant reduction in flow-through costs. Moreover, the irregular cadence of the SAM & PM programs has resulted in human and financial management challenges for industry partners and universities researchers, and uncertainty among international partners regarding Canada's financial commitments to ongoing and proposed missions. The enviable reputation that Canada has established over many years with its signature contributions to international joint ventures and high-profile space exploration missions may have been affected over the years due to the lack of investments. [Findings: 17, 18]
6. Management response and action plan
Organisation / Function
|Management response||Details of action plan||Schedule|
|Recommendation # 1|
|The program should develop clear guidelines that clarify the definition and scope of science support to missions in order to allow for the continuity of science support funding throughout all phases of a mission from pre-definition phase to post-operation activities.||Space Exploration in collaboration with Space Utilization, Programs and Integrated Planning and Finance||Management agrees with the recommendation.||
Clarify the definition and scope of science support throughout the different phases of science missions. Identify proper mechanisms to fund science in space astronomy and planetary exploration missions throughout the evolution of the mission from pre-Phase 0 to Phase F. This will require clarification of the ownership of data for science missions.
Investigate the inclusion of science management plans as a deliverable in the Investment Governance and Management Framework (IGMF) to define the approach and costing of science throughout the full mission cycle, from Phase A all the way to data reduction and analysis in Phase F.
Assess feasibility to ensure science support is well covered throughout the gating process of the IGMF.
Appendix A: SAM & PM logic model
Space Astronomy & Planetary Missions Logic Model
(PAA 126.96.36.199 & 188.8.131.52)
Source: Performance Measurement Strategy for Space Astronomy Missions and Planetary Missions –
Appendix B: Case studies
Case study: Mars Science Lab – Alpha Particle X-Ray Spectrometer (APXS)
|Mission title:||Mars Science Lab – Curiosity Rover|
|Space Agency Partner||NASA|
|CSA Program||Planetary Missions|
|Duration; Start–End Dates||–present|
|CSA Funding||~ CAN$20 million|
|Space Agency Partner Funding||~ US$2.5 billion|
Project background and overview
The NASA Mars Science Laboratory (MSL) is one of a series of scientific missions to Mars using rover technology, in this case, the Curiosity rover. It was launched on and successfully landed on Mars on ; the mission has been extended twice for two-year periods, remains in operation and is expected to remain operational until at least . MSL was designed to assess whether Mars ever was, or is still today, an environment able to support microbial life. This includes characterizing the geology, climate and the conditions that might have at one time supported life on Mars. There are 10 state-of-the-art science instruments on the Curiosity rover, including Canada's Alpha Particle X-Ray Spectrometer (APXS).
The Curiosity rover acts as a robot geologist and has a robotic arm with the APXS attached; the APXS is used to analyze the chemical composition of selected rocks and soil by employing X-ray spectroscopy. Its purpose is to "
map the vertical and lateral distributions of major, minor and trace elements in order to connect the results with the geological context to produce an understanding of the formation, evolution and biological potential of the landing site environment."Footnote 53 The Gale Crater landing site near Mount Sharp was chosen to investigate the sedimentary layers that preserve the history of billions of years of climatic conditions on the planet.Footnote 54 The APXS has been used in a triage function several times a week, during which the rover is moved as a first step before deciding whether to drill into selected rocks to take a sample for analysis by the other key scientific instruments. It has been instrumental to the success of the missionFootnote 55 through the elemental abundance measurements it has taken to find traces of salts such as chlorine, bromine, and sulfur mixed with oxygen into sulfate that have collected into deposits and are indicative of where there has been water. It has thus contributed to most of the mission's major findings, including the first-ever age dating of rocks from the surface of Mars and confirmation that that the area around Mount Sharp was once a freshwater lake with associated rivers and groundwater that might have supported life in the distant past.Footnote 56,Footnote 57
The Canadian contribution to the MSL is, however, not just a hardware contribution limited to the APXS scientific instrument, but rather it is what is known as an instrument investigation. The instrument investigation includes an entire team of Canadian scientists who are actively participating in mission operations on a quasi-daily basis and analyzing data to produce scientific results in line with mission objectives. Canadian members of the APXS instrument investigation team include the Principal Investigator (PI) and two Co-Investigators as well as several postdoctoral fellows and graduate students. Their expertise is relied upon to operate the APXS, to analyze and interpret the data produced by the APXS and to communicate the results to the rest of the MSL team.
In addition to the APXS investigation, Canada also supports three Mars Science Laboratory Participating Scientists from Canadian universities. These positions are competitively selected by NASA and the CSA approximately every three years through an open, competitive process, and the incumbents hold the equivalent of Co-Investigator roles on the mission. One of the participating scientists works primarily with APXS data. The other two work primarily with ChemCam and REMS, two of the remaining nine instrument payloads.
Continued need for the program
The continued need for the planetary exploration program has to be understood within the context of the Canadian space community. The strength of this community is based in the tri-part relationship between government, represented primarily by the Canadian Space Agency, the space industry sector and academia. There are small, medium and large companies within the space industry and among them are satellite operators, those focused solely on Earth observation, satellite communications, space sciences, navigation, and then those involved in space exploration. Canada's space community is also quite broad in terms of the domains of expertise that exist within Canadian universities and research centres, including human, health and life sciences, geophysics and astronomy. The movement of highly qualified personnel (HQP) from the space industry to universities and vice-versa has been noted in the interview data.
The planetary exploration community is a relatively new component of the space community. Based on CSA research and interview data, there was one person in Canada engaged in planetary missions at the turn of the millennium, with a small number of additional researchers who worked on fundamental research in dynamics modelling, meteorite analysis and terrestrial impact cratering, areas that didn't require access to space.Footnote 58 Since then, the community has grown to the currently estimated 26 faculty members, who hold key research positions in top Canadian universities and who have been or are currently science team members (Co-Investigators) in planetary exploration missions. The data gathered through CSA group interviews reveals that much of this growth occurred during the evaluation time frame from to and can be attributed to the CSA's participating scientist program, the mission Co-Investigator program and other ways for scientists to participate in joint missions with partner space agencies.
All e-surveyed funding recipientsFootnote 59 rated the need for the continued funding of planetary exploration programs as either significant or very significant to their research. While the selection of scientists to participate in international space agency planetary exploration missions is competitive and merit based, the financial contribution of the Government of Canada via the CSA is a proposal selection requirement. Without this support, the opportunities for involvement as a Co-Investigator would be severely limited, and Canadian planetary scientists would have to rely on archived data available through open access platforms, such as NASA's Planetary Data System, to conduct their research. Based on the CSA group interview data, this prospect has prompted several prominent scientists to openly discuss leaving Canada, while some young, talented scientists have already left. From the industry perspective, involvement in planetary exploration missions supports Canadian industry competitiveness in the international space market by positioning companies such as MDA as a reliable developer of operational space exploration instruments.
The continued need for the program was also assessed from the perspective of the CSA's partners. When the MSL mission was extended by NASA in and Government of Canada financial support for the APXS was under review, the NASA lead scientist wrote, "
It is because our trust in CSA as a reliable partner and the scientific utility of APXS that we selected this instrument as an important integral contributor to the mission's scientific success."Footnote 60 The review of documents related to multiple extensions of the mission also confirmed that the APXS was strongly relevant to NASA and the Jet Propulsion Laboratory operating the Curiosity rover, given its very important function on the mission, the contribution of Canadian scientists and the expectation of continued financial support.Footnote 61,Footnote 62,Footnote 63,Footnote 64 Withdrawal of support for the APXS for the extension phases of the MSL would have resulted in the crippling of the Curiosity rover science operations, the loss of Canada's credibility and reputation as a reliable partner, the loss of opportunities to develop more HQP and the loss of planetary sciences research.
Effectiveness – Outcomes
Better positioned for space opportunities
The Space Exploration Program, including Planetary Missions, is expected to contribute to Canada's strategic positioning to influence space exploration missions and decision-making.Footnote 65 NASA's Announcement of Opportunity for the MSL was a competitive call for science instruments that would best meet the science objectives of the mission. The Canadian APXS proposal was based on a new university, industry and government partnership between the Principal Investigator, Professor Ralf Gellert at the University of Guelph, the prime contractor, McDonald, Dettwiler and Associates (MDA), and the CSA. NASA selected the APXS in late as a Principal Investigator-led investigation and the financial contribution was approved in at initial cost of CAN$9.8 million. The instrument was delivered to NASA within budget in , well in advance of the launch date.Footnote 66
Aside from playing an important role in preparing the scientific components of the successful proposal, the University of Guelph provided the scientific direction for the design and engineering support for the development of the instrument, calibrated and supervised the testing of the instrument and has led the science operations, which includes directing the instrument and data analysis.Footnote 67 This was a reduced role relative to NASA's expectations, where the NASA competition called for the Principal Investigator to have overall authority for the scope of the instrument development. Principal Investigator decision-making authority over scope and risk is not consistent with CSA project management practice, and is considered a risk if a university does not have the staff and experience to manage a space project. At the time of selection only one other Canadian scientist was included in the team, Dr. Iain Campbell, also from the University of Guelph. In addition to the role of funder, the CSA assumed the lead role of Instrument Engineer and Technical Operations in direct liaison with the MSL project manager. In two later selection processes (a space science enhancement competition in and participating scientist competitions in and ), other Canadian scientists were integrated with the MSL Science Team, including researchers from McGill University, Brock University, York University, the University of Western Ontario and the University of New Brunswick,Footnote 68 which all received funding from the CSA for their initial and ongoing involvement in the mission.Footnote 69
There were three earlier versions of the APXS on previous Mars rovers: the Pathfinder, Spirit and Opportunity rovers that came out of the Max Planck Institute for Chemistry in Mainz, Germany. After encouragement by MDA, Professor Gellert brought the APXS to Canada when the German institute closed down, and the CSA supported the move of Professor Gellert to the University of Guelph (UoG), where he became the Principal Investigator. This was the first time Professor Gellert had participated in a Mars instrument selection process as a Principal Investigator. In the Max Planck institute, he developed his expertise through building and testing APXS as a researcher for the former Principal Investigator, who had retired. Based on the interview data, MDA played a lead role in positioning the APXS with NASA and the CSA for the MSL mission. Furthermore, the interview data revealed that it can take up to five years for a company to position itself to present a proposal for an opportunity such as this. The proposal was submitted along with an endorsement letter from the CSA stating that "
Should NASA select the APXS instrument for flight on the MSL mission, it is the intention of the CSA to consider funding work as described in the proposal."Footnote 70 MDA became the prime contractor for the APXS; that is, it was the instrument developer in – and then provided standby technical support in –. It should also be noted that MDA Information System Space Division eventually bought the US company that built the robotic arm on the Curiosity rover, further positioning itself for future planetary exploration missions.
The next opportunity that presented itself was NASA's Mars mission, designed to discover evidence of past microscopic life on the planet. Based on the collected group interview data, NASA had expressed an interest in a possible Canadian contribution to the mission in the form of a science instrument or engineering subsystem. The APXS was repeatedly referred to in the Science Definition Report and was included among the scientific instruments in the "
strawman" or hypothetical payload. Based on corroborating interview data from different stakeholders, the CSA recognized the importance of this mission to the Canadian planetary science community and through recommendations made by the Planetary Exploration Consultation Committee. The CSA then published a competitive Request for Proposal that the MDA/UoG APXS team won, and in it went on to submit their costed proposal of approximately CAN $12 million to NASA for the Mars competition. However, the proposal was submitted without an endorsement letter assuring financial support from the CSA, as the Government of Canada approvals were not forthcoming. Although Canada was given every opportunity to participate in what might be an historic mission to Mars that may discover evidence of life on another planet, NASA was obliged to select another scientific instrument from another country with slightly different capabilities. The opportunity to secure Canada's preferred partner status with NASA for the Mars mission was not pursued.
Technology transfer enhanced
The Space Exploration Program, including Planetary Missions, is expected to contribute to the transfer of scientific knowledge and know-how for multiple uses and applications;Footnote 71 this includes the transfer to terrestrial applications and reuse in other space missions. In the case of the APXS, the opportunities for geological prospecting on Earth are limited because it was designed to work in a vacuum with not much atmosphere similar to that on Mars, which has 1/100 of the Earth's atmospheric pressure. Based on the interview data, the radioactive elements in the instrument also preclude its common use for geological applications on Earth, although the techniques for interpreting the data are considered viable. While the interview and the e-survey data support the notion that space technology can have commercial applications for use on Earth, the spin-off applications are often unexpected and occur long after the space missions take place. The reverse, however, is more common, where terrestrial technologies are adapted to the needs of space scientists and put through a qualification process for use on missions. Nonetheless, three of the four e-survey respondents associated with the APXS declared that their experience had enabled them to transfer their newly acquired expertise to other applications on Earth.
The history of the APXS clearly illustrates the benefits of continued participation in space exploration missions with the same science instrument. NASA's MSL lead scientist and the Canadian APXS principal investigator both commented on the invaluable APXS contribution in terms of scientific data continuity about the planet's geology. "
An added advantage of APXS is that there is a body of similar measurements from other Mars missions that form a reliable basis for regional comparisons."Footnote 72 During the group interview, one interviewee commented that "
the value of the APXS chemical data is most obvious when they are not there." Space industry investments in space technology are based on the business case point of view that the technology will be reused in future space exploration missions. The interview data revealed that despite the recent setback with Mars , and now the increased competition from similar science instruments, there is still considerable optimism about the potential of APXS being reused in future space missions as it is suitable for research on asteroids, comets and other planetary bodies. E-survey respondents associated with the APXS also indicated that their involvement has had a positive impact on their position to respond to and win other opportunities to undertake planetary exploration work.Footnote 73
Contribution to sustained economic growth
The expectations that Canadian involvement in space exploration and the development of innovative space technology would contribute to sustained economic growth for the Canadian space industry have been a part of the rationale for funding for approximately a decade. The evidence to support this notion has proven to be difficult to obtain when measured in quantifiable terms, such as the value of follow-on contracts, the value of commercial sales, and the number of full-time equivalent jobs created for HQP resulting from the commercialization of space technology.
The available data for this case study is for the most part anecdotal but still convincing. Half of the e-survey respondents associated with the APXS acknowledged having received follow-on research grants from NSERC, the Ontario Ministry of Research and Innovation, and other CSA programs thanks to their involvement in planetary exploration missions.Footnote 74 The APXS has no terrestrial commercial value, however, and didn't result in the creation of any additional jobs, other than those associated with the science operations of the instrument itself. Based on the interview data, the University of Guelph Planetary Observation Centre responsible for the science operations of the APXS employs the Principal Investigator, an electronic engineer, a program manager, a postdoc and several graduate students. When the personnel from other universities are included, the total amounts to about 10 to 15 people working on related contracts.
For the space industry partners, the instability in the planetary exploration program has made human resource, revenue stream and overall financial management very challenging given the cadence of small planetary missions and the occasional large space exploration contract. Nevertheless, based on the interview data, the value of continued involvement is in remaining current in the latest technology developments, the potential spin-in and spin-off opportunities of those technologies, the ability to attract HQP and the enhanced credibility that comes with having developed scientific instruments and robotics for space research on asteroids, Mars and potentially the moon.
Canada's enhanced profile and visibility
The cost of NASA's MSL mission is over US$2.5 billion including the extensions in mission operations, while the Government of Canada's support of the APXS has cost in total approximately CAN$20 million. Although the APXS is a small and relatively low-cost science instrument, it is nevertheless a vital component of the MSL payload. As 1 of 10 payloads, it provides a leadership role in MSL mission operations to Professor Gellert through the MSL Project Science Group. The highly successful MSL mission has been very productive in generating scientific data on the elemental composition of the soil and rocks in the region of Gale Crater and, according to NASA's lead scientist, a significant portion of that production is attributable to the APXS.Footnote 75 Canada's reputation with NASA as a trusted and reliable partner in space exploration has been enhanced because of the contribution of Canadian scientists and technology to the mission.Footnote 76
The iconic image of the Curiosity rover as it moved across the Martian terrain has made the MSL mission highly visible and recognizable to a broader general public audience. NASA's attention and investments in public communications have also enhanced the visibility of the Canadian contribution, as Canadian news services and public media pick up the available press releases and images. This was particularly evident during the landing of the Curiosity rover almost five years ago. The contribution that the APXS has made to enhancing Canada's international profile and visibility in space exploration was also confirmed by three out of the four respondents to the e-survey related to the APXS as having exceeded expectations.Footnote 77 However, the interview data suggest that the CSA could have done much more to communicate the nature and importance of the APXS contribution to the Canadian public, and particularly to inspire children and young adults to pursue careers in science. Resource constraints were cited in the interview data as having hampered the CSA's capacity to implement a general communications and outreach strategy. For example, while most of the other instrument teams on MSL have active websites to promote activities, there is still no APXS website at the University of Guelph.
In , there was some question as to whether the Government of Canada would continue to fund the APXS for the proposed two-year extension. The document review revealed that a loss of credibility and reputation was invoked in making the business case to support this extension, as well as those that followed.Footnote 78 Several letters of support from leading scientists in Canadian universities also emphasized the importance of the mission for advancing scientific knowledge about the planet Mars, the important role that the APXS has played, and the risk to Canada's future participation in such missions if a decision to withdraw was taken. The incertitude of the Government of Canada's financial support for the APXS with each successive extension of the MSL mission and the decision not to support the Mars proposal was received with disappointment, but as one senior NASA official put it, "
If partnerships don't materialize with Canada or Europe, somebody else will fill the vacuum." In fact, this Mars decision led to a new, competing capability being established in the US, as another instrument was picked to provide APXS-like composition measurements.
Case study: James Webb Space Telescope – Near Infrared Imager and Slitless Spectrograph (NIRISS) and Fine Guidance Sensor (FGS)
|Mission title:||James Webb Space Telescope|
|Space Agency Partner||NASA|
|CSA Program||Space Astronomy Missions|
|Duration; Start–End Dates||–present|
|CSA Funding||~ CAN$214 million|
|Space Agency Partner Funding||~ US$8.8 billion|
Project background and overview
Although the original idea for an eight-metre, passively cooled, near-infrared telescope in a high Earth orbit was first voiced at a workshop held at the Space Telescope Science Institute, the report to propose such a successor to the Hubble Space Telescope originated in with the American Association of Universities in Astronomy.Footnote 79 It was first known as the Next Generation Space Telescope and renamed in to the James Webb Space Telescope (JWST), after the NASA administrator responsible for the Apollo Program. The JWST mission is an international collaboration led by NASA in partnership with ESA and the CSA. In , the CSA and NASA signed an Agreement for Cooperation on the James Webb Space Telescope, under which the CSA agreed to develop and deliver the FGS/NIRISS instrument, and to provide functional support to the science operations of JWST. JWST is the most ambitious and prestigious space astronomy mission ever undertaken, designed to study "
the shape and chemical composition of the universe, the origin and evolution of galaxies and stars, and the nature of the unseen dark matter."Footnote 80 JWST will view images in the infrared spectrum using four science instruments, including the Canadian Fine Guidance Sensor (FGS) and the Near Infrared Imager and Slitless Spectrograph (NIRISS). Based on the interview data, the FGS is a very sensitive infrared tunable camera with moving target tracking capability that enables the telescope to be aimed very precisely; it is the steering wheel of the observatory and considered a mission-critical instrument. The NIRISS is a general-purpose infrared camera with unique capabilities for exoplanet detection and atmospheric studies.Footnote 81 The FGS/NIRISS flight instrument was delivered to NASA in , and has since been integrated with the optical telescope and went through several campaigns of environmental tests.
The JWST mission has had a history of cost overruns and delays and is now one of the most expensive scientific instruments ever built in the world, at over US$9 billion. The overall cost to NASA is now estimated to be approximately US$8.8 billion.Footnote 82 In , the Government of Canada approved CAN$71.7 million for the preliminary project design. Once the design was completed two years later, an additional CAN$104.2 million was approved, with an anticipated launch date of .Footnote 83 However, for a variety of reasons, including the faulty infrared detectors manufactured by a foreign supplier and the underestimated time required for mission integration and testing, a NASA re-planning exercise resulted in an extension of the project life by 5.5 years, with the current launch date scheduled for -.Footnote 84 The overall cost of Canada's contribution to the JWST mission escalated to an estimated CAN$214.7 million to deliver the FGS/NIRISS and provide science support through to mission completion in .
The Canadian contribution to the JWST mission is much more than just the design, development and delivery of the two science instruments; it includes Canadian personnel working in various mission capacities, such as scientists, data analysts, historians and members of the JWST science working group. There are also at least four Canadian astronomers working at the Space Telescope Science Institute in Baltimore, Maryland, which is the current science operations centre for the Hubble Space Telescope and the future operations centre for the JWST.
Continued need for the program
The JWST mission is fully aligned with the CSA's mandate and strategic outcomes, specifically the expansion of scientific knowledge acquired through space exploration, as well as the application of newly acquired knowledge and know-how. The continued need for the Space Astronomy Missions program should, however, be understood within the rich historical context of the astronomy community in Canada. A quick review of the Canadian Astronomical Society website and history page is revelatory; it is worth citing that the first amateur astronomical organization in Canada was formed in 1867 and named the Royal Astronomical Society of Canada. While the Canadian astronomy community expanded slowly relative to its American counterpart, the building of ground-based observatories was instrumental to the scientific research undertaken and the continued growth of the community. Of note were the Dominion observatories; the University of Toronto David Dunlap Observatory that opened in with the world's second-largest reflector; the support of the National Research Council's radio observatory; and later the Herzberg Institute of Astrophysics. By , when the Canadian Astronomical Society was established for professionals, it attracted approximately 100 members. Based on the interview data, today it is estimated that the astronomy community numbers 500–600, of which approximately 300 are professional astronomers and the balance are graduate and postgraduate students in universities across Canada. The Canadian astronomy community is among the most influential in the international community, based on standard bibliometric measures of scientific performance and impact; it was at the top of the rankings between – of all countries that contributed to scientific advances in the field of astronomy and astrophysics.Footnote 85
All e-surveyed funding recipients rated the need for the continued funding of the space astronomy program as either significant or very significant to their research. This funding, at a fraction of the total cost of leading-edge space astronomy missions, is viewed as essential if Canadian scientists are to play a significant role in mission operations, benefit from having a Canadian science instrument on board, and secure Guaranteed Time Observation (GTO). The latter is reserved for only those scientists who make significant contributions in terms of key hardware and software components, such as the Principal Investigators of the four science instruments onboard JWST. Of the general guest observation (GO) time, 450 hours of the GTO, or at least 5%, is expected to be allocated to the Canadian astronomical community.Footnote 86 In , the JWST GTO team received observing proposals from the 20 GTO allocation holders for Cycle 1 observations, including the Canadian "
NIRISS Exploration of the Atmospheric Diversity of Transiting Exoplanets," the "
NIRISS Survey for Young Brown Dwarfs and Rogue Planets," as well as several others.Footnote 87 The guaranteed opportunity to direct the world's most powerful space telescope would not be possible for Canadian astronomers without the financial contribution from the Government of Canada to the JWST mission. In addition, other Canadian astronomers, including postgraduate doctoral fellows, will also be well positioned for success in the open call for proposals reserved for General Observers, thanks to their involvement directly or indirectly with the FGS/NIRISS science teams and their familiarity with the JWST capabilities. Review of the available documentation strongly suggests a high-impact use of the GTO allocation to Canada that will lead the Canadian astronomical community to maximize the use of the JWST, further advance their scientific knowledge, possibly lead to new discoveries by Canadian scientists and further enhance Canada's reputation in the field of space astronomy.Footnote 88,Footnote 89
As frequently stated in the available documentation, although not confirmed with interview data from an industry perspective, involvement in the JWST mission supports Canadian industry competitiveness in the international space market and creates highly skilled jobs, high-tech exports, and commercialization spinoffs.Footnote 90 The continued need for the program was assessed from NASA's perspective in , when the project prepared for a preliminary design review and the temporary suspension of additional funding was implemented. Sustained and uninterrupted support was viewed as essential at such a critical stage of development and the CSA's central role in JWST was a testament to the very successful history of cooperation between the two space agencies.Footnote 91 The interview data show that a few years later when delays and cost overruns threatened the cancellation of the mission by the US Congress, it was apparently helpful to have supportive international partners involved. In the most recent assessment based on the interview data, Canadian involvement exceeded NASA's expectations, not only because the FGS met all of its requirements and the NIRISS redesign was a success, but also because the CSA's contribution was a "
model of collaboration." Canada's reputation as "
one of NASA's most valued international partners" has undoubtedly been maintained as a result.Footnote 92
Effectiveness – Outcomes
Better positioned for space opportunities
The Canadian contribution to the JWST mission has been a collaborative effort between government represented by the CSA and the National Research Council Canada (NRC), industry partners represented by the prime contractor COM DEV Inc. (COM DEV), and universities from across the country. The composition of the FGS/NIRISS science team reflects the CSA's attention to the regional distribution of involvement in major crown projects and overall outreach to established partners.Footnote 93 The Principal Investigator of the FGS/NIRISS is located at the University of Montréal and works in collaboration with astronomers and scientists located at the University of Toronto, the University of Alberta, Saint Mary's University in Halifax, University of Ottawa, the NRC's Herzberg Institute of Astrophysics in Victoria, as well as Canadian scientists in Switzerland and the United States.Footnote 94 The funding provided to support the Principal Investigator's involvement positioned the University of Montréal to attract additional funding to establish an exoplanet research institute that in fall will begin hiring a number of students, postdoctoral scholars and fellows to analyze the NIRISS observation data. This Institute is well positioned to make some very important discoveries—for example, probing the atmospheres of exoplanets to determine the presence of water, habitable environments and, as one interviewee noted, "
being able to point to an object in the sky and say, 'There could be life there!'"
The role of the industry partners, in this case COM DEV, has been to design and build the FSG/NRISS in accordance with the requirements and guidance provided by the Canadian science team. In , COM DEV had purchased the Space Science and Optical Instruments Division of the original prime contractor EMS Technologies Canada Ltd. and was thereafter assigned the prime contract. In , the CSA commissioned a costing and capability assessment to determine if COM DEV could complete the Phase D flight model build and test as per the proposed cost and schedule. It was determined that COM DEV's cost proposal was reasonable and that it had the requisite expertise, management and engineering skills at its Ottawa facility and could draw on the considerable resources of its Cambridge, Ontario, facility for the manufacturing and testing of the optical components.Footnote 95 By , the total value of the prime contract had risen from the estimate of CAN$82.6 million to approximately CAN$154 million.Footnote 96 It took longer to complete and deliver the FGS/NIRISS science instruments than originally planned due to the delays incurred by NASA, such as changes in scope requirements requested by NASA, faulty infrared detectors supplied by a foreign supplier, and the need to undertake a complete redesign of the FGS/NIRISS one year before the delivery deadline. COM DEV had employed at least a dozen HQP on the project—and up to 50 HQP in the peak periods of designing, building and testing of the flight hardware prior to delivery to NASA—who would otherwise not have been employed with the company. This strengthened its human resource capacity and further positioned it to take advantage of any potential spin-offs from the technology that it had built and delivered, although the science instruments themselves belong to the Crown.Footnote 97 Furthermore, having delivered a mission-critical component for such a high-profile and prestigious undertaking as the JWST clearly enhanced its credibility and reputation as an international space hardware and systems business; COM DEV was purchased in by the American multinational conglomerate Honeywell International Inc., which merged it with the company's Defense and Space business line.
Under the Memorandum of Understanding with NASA, the CSA is responsible for providing the FGS/NIRISS instrument to NASA. As a result, it manages the contract with COM DEV to develop and deliver a qualified FGS/NIRISS instrument to NASA, and to support the NASA integration and test activities for the observatory integration and test. From late to , it had a project team member at COM DEV's facilities in Ottawa, Ontario, to ensure that COM DEV was fully aware of the project requirements and would satisfy them. While the position was no longer required after the delivery of the flight hardware to NASA, the oversight responsibility continues until the completion of the on-orbit deployment of the telescope.Footnote 98
Technology transfer enhanced
The Space Exploration Program, including Space Astronomy Missions, is expected to contribute to the transfer of scientific knowledge and know-how for multiple uses and applications.Footnote 99 This includes the transfer to terrestrial applications and reuse in other space missions. In the case of the JWST, the expectation for industry was that involvement would stimulate spin-off technologies and business opportunities, as well as position COM DEV globally for future international collaborations in future space exploration.Footnote 100 The interview and e-survey data, however, indicated that there were no known terrestrial applications for the technology, perhaps because it was still too soon or because the two science instruments were over-designed to operate in a cryogenic environment with zero maintenance requirements.Footnote 101,Footnote 102 New competencies were to some extent acquired by the scientists involved, and future data analysis tools and know-how are expected to be developed.
Positioning Canada for future space exploration missions was noted in the foundational documents. The rationale for involvement in JWST is that it would support Canada's ability to influence decision-making processes in international fora regarding priority missions.Footnote 103 Given that NASA wanted to source the guider function from Canada for JWST because of past demonstrated expertise and technology, based on the interview data, the potential for future collaboration with NASA and other space agencies has been deemed quite high now that the flight hardware has been successfully delivered.
Contribution to sustained economic growth
The expectations that Canadian involvement in space exploration and the development of innovative space technology like the FGS/NIRISS would contribute to sustained economic growth for the Canadian space industry has been a part of the rationale for funding for approximately a decade. The evidence to support this notion has proven to be difficult to obtain when measured in quantifiable terms, such as the value of follow-on contracts, the value of commercial sales, the number of full-time equivalent jobs created for HQP resulting from the commercialization of space technology, and so forth. The evaluation did not find any evidence of sustained economic growth accrued to COM DEV subsequent to or concomitant with the development and delivery of the FGS/NIRISS space hardware. However, at least a dozen HQP were employed by COM DEV who very probably will apply their accrued knowledge expertise and know-how to other space-related projects for Honeywell or other Canadian space industry companies. The University of Montréal will also employ additional HQP to analyze the NIRISS data, developing expertise that would not otherwise be possible. The CSA will also provide grants for analysis of the JWST data during the mission that will result in the creation of several postdoctoral fellow positions in Canadian universities. While the retention and attraction of HQP in industry and academia is evident,Footnote 104 the prospects for contributing to sustained economic growth are indirect, not very quantifiable and highly dependent on the health of the space industry in Canada.
Canada's enhanced profile and visibility
The estimated cost of the JWST mission to NASA is currently US$8.8 billion, while the Government of Canada's support of the FGS/NIRISS has cost to date a total of CAN$214 million. Despite the escalation in cost and technical setbacks along the way, the CSA and the Canadian astronomy community have demonstrated their capability to deliver a mission-critical component for one of the most prestigious and complex scientific instruments ever built. The interview data suggests that the success of this endeavour, although not yet completed, can be attributed to the collaboration between government, industry and academia. The data also suggests that it was not the strength of the optical and infrared sensing technology, which is not proprietary to Canada, that was the key factor, but rather the knowledge, expertise, know-how and collaborative effort demonstrated by those involved. JWST has enjoyed tremendous support from the Canadian science and astronomy community and was characterized as "
an affair of the whole nation," as one of the interviewees suggested.
According to a senior NASA official, the FGS/NIRISS may be Canada's most important space science contribution to date, as "
it enhanced the capability of the telescope" and made it more accessible to worldwide use. The reputation that Canada seems to have developed, aside from being a model of collaboration, is that it has contributed more than one might expect from the amount of dollars that have been available to invest. JWST has undoubtedly contributed to Canada's very strong credibility among its international partners for what it can deliver for the resources available.
- Date modified: