Evaluation of the Enabling Technology Development Program of the Canadian Space Agency

For the period from April 2010 to March 2015

Project # 15/16 – 02-01

Prepared by the Audit and Evaluation Directorate

June 2016

Table of contents

  1. Introduction
  2. Background
  3. Evaluation approach and methods
  4. Results
  5. Conclusions and Recommendations

List of tables and figures

Acronyms used in the report

  • AO: Announcement of Opportunities
  • AETD: Advanced Exploration Technology Development
  • CSA: Canadian Space Agency
  • EOADP: Earth Observation Application Development Program
  • ESA: European Space Agency
  • FAST: Flights and Fieldwork for the Advancement of Science and Technology
  • FTE: Full-time equivalent
  • GPS: Global Positioning System
  • GRIP: Government Related Initiatives Program
  • G&C: Grants and Contributions
  • G&C COE: Grant and Contribution Centre of Expertise
  • HQP: Highly Qualified Personnel
  • IP: Intellectual Property
  • IPMTT: Intellectual Property Management and Technology Transfer
  • IRC: Industrial Research Chair
  • ISS: International Space Station
  • NASA: National Aeronautics and Space Administration
  • NSERC: Natural Sciences and Engineering Research Council of Canada
  • O&M: Operations & Maintenance
  • PAA: Program Alignment Architecture
  • PSPC: Public Services and Procurement Canada
  • RCM: RADARSAT Constellation Mission
  • RFP: Request for Proposals
  • R&D: Research and Development
  • SADI: Strategic Aerospace and Defence Initiative
  • SS&T: Space Science and Technology
  • STDP: Space Technology Development Program
  • STRATOS: Stratospheric Balloon Program
  • SSP: Sub-Sub-Program
  • TDP: Technology Demonstration Program
  • TDR: Technology Development Request
  • TPWG: Technology Planning Working Group
  • TRL: Technology Readiness Level

Executive Summary

Background

This report presents the findings of the evaluation of the Enabling Technology Development Sub-Sub-Program (SSP), which covers the activities of the Space Technology Development Program (STDP) and the services offered by the Intellectual Property Management and Technology Transfer (IPMTT) group.

The STDP is an ongoing research and development (R&D) program of the Canadian Space Agency (CSA) that provides financial support through contracts and contribution agreements to entities that are selected based on their capacity to develop specific space technologies. The two fundamental and interrelated goals of the STDP are to formulate, implement, and manage R&D contracts related to the development of mission-enabling and generic technologies to support future needs of the Canadian Space Program; and to support industrial capability-building through the development of new products and services, processes, and know-how.

The IPMTT group is responsible for the management of Crown-owned technologies and licences, as well as of technology transfers. It also provides consultation services and training sessions on a range of IP matters, such as R&D contracts and communications. In doing so, it works collaboratively with the CSA's three program sectors (Space Utilization, Space Exploration, and Space Science and Technology); CSA corporate functions (Legal Services, Information Management, and Finances); other federal departments, such as Public Services and Procurement Canada (PSPC); and Canadian space industry and academia.

The evaluation study was conducted by PRA Inc. between September 2015 and March 2016 on behalf of the CSA Audit and Evaluation Directorate, and it covered the period from April 2010 to March 2015. The evaluation is a requirement of the CSA five-year evaluation plan and was conducted in accordance with the Treasury Board's Policy on Evaluation (2009), which requires that all federal programs be evaluated every five years. An Evaluation Consultative Group, consisting of representatives from both the STDP and the IPMTT groups, provided guidance and support throughout the evaluation.

The evaluation involved the following methods: a review of relevant literature, documentation, and program data; 19 interviews involving 31 individuals from CSA senior management, the STDP, the IPMTT group, CSA internal clients, CSA Grant and Contribution (G&C) Centre of Expertise, PSPC, and industry and academia; and three case studies that involved a document and data review, as well as seven interviews with 11 individuals.

Conclusions and Recommendations

Relevance

The development of new technologies stands at the core of CSA's ability to actively engage in space activities. Throughout the period covered by the evaluation, a number of reports and studies have urged the federal government to enhance the CSA's ability to carry out space technology development in a more systematic and predictable manner. It was also recognized that such a goal can only be achieved through the direct involvement of a strong and sustainable Canadian space industry.

While the CSA has a number of programs that support space technology development, the STDP continues to be the main tool through which space technologies related to future space mission opportunities are developed. Without the STDP, it is unlikely that the CSA would be in a position to maintain its current involvement in space activities. It is also through the STDP that the Canadian space sector can access the type of support it requires to develop technologies that enhance its industrial capability. As much as this statement speaks to the relevance of the STDP, it also illustrates the fact that the federal government has yet to implement a strategy that would engage a broader range of federal partners in supporting space technology development. Programming offered by other federal departments or agencies either remains limited in scope, or does not easily accommodate the nature of space technologies.

The predominant role that the private sector plays in conducting space technology development reflects a historical trend within the CSA; it also reflects practices in other space agencies around the world. Over the years, only limited R&D activities have been undertaken within the CSA. During the period covered by the evaluation, the CSA essentially ended these activities as a result of a strategic review process. Early signs indicate that this approach may prove problematic over time, as in-house subject matter expertise is not being renewed. It is indeed essential that the CSA maintains its ability to monitor and support the work done through procurement processes.

Recommendation #1: The CSA should explore avenues to sustain the level of in-house expertise required to ensure the proper management of space technology development.

The SSP also provides the framework needed to manage the range of IP requirements associated with space technologies. It is recognized that only through an effective management of IP can the CSA ensure that the technologies it supports benefit the CSA and the broader Canadian society.

Program design

The SSP benefits from a sound program delivery structure covering both STDP and IPMTT group activities. For the two components, the CSA has established efficient processes and structures that have proven capable of undertaking the range of activities covered by their respective mandates.

As it relates more specifically to the STDP, it must be acknowledged that the absence of a long-term plan establishing space priorities is limiting the ability of the program and its internal clients to plan technology requirements. More specifically, the evaluation confirms that the STDP is capable of undertaking the research required by internal clients, and the Technology Planning Working Group (TPWG) has proven useful in that regard. But the broader question of whether the STDP will deliver the required technologies at the appropriate time to allow the CSA to implement its broader agenda cannot be fully addressed in the absence of such plan. This issue, while certainly relevant to the STDP, does fall beyond the scope of this evaluation.

Since processes that lead to the issuance of Requests for Proposals (RFPs) for research contracts and Announcements of Opportunities (AOs) for contributions involve multiple steps and internal stakeholders, it can be challenging for the STDP to plan with precision the release of these RFPs and AOs and, thus, inform external partners who may be in a position to respond and submit bids or proposals. Recognizing these factors, it must be noted that any gains that could be achieved in making these releases more predictable and easier to manage for external stakeholders would benefit the program.

Recommendation #2: The CSA should explore avenues to enhance the planning and communication activities leading up to the releases of Requests for Proposals and Announcements of Opportunities, in order to ensure predictable funding timelines and enhance the ability of the Canadian space community to respond to these calls.

The efforts initiated during the period covered by the evaluation to consolidate a portion of space technology development carried out by the CSA will provide an opportunity to clearly establish how the STDP is positioned in relation to other space development programming. To this end, the CSA should ensure that its Program Alignment Architecture (PAA) and the associated reporting more clearly reflect the range of activities undertaken by both the STDP and the IPMTT groups, the latter being absent from the current PAA.

Recommendations #3: The CSA should review its current Program Alignment Architecture (PAA), in order to appropriately reflect the range of activities undertaken by the STDP to support generic and mission-enabling technologies, as well as enhancing the industrial capability of the space sector. The CSA should also ensure that the next PAA revision appropriately reflects the range of activities undertaken by the IPMTT group.

The STDP and the IPMTT groups have developed performance measurement strategies that provide helpful information on the activities they undertake and the results they are aiming to achieve. This evaluation has identified a number of avenues that could be explored to enhance these strategies. In particular, the CSA should clarify how it intends to report the performance information it is expected to gather.

Recommendation #4: The CSA should review the current performance measurement strategies for the STDP and the IPMTT groups, based on the opportunities for improvement identified in the evaluation report, including strategies to more effectively communicate actual SSP impacts and how they contribute to the CSA mission.

Performance

The two components of the SSP have successfully implemented the range of activities required by their respective mandate. The STDP group has launched several new research contracts, in addition to managing ongoing R&D projects and implementing the contribution components of its mandate. The IPMTT group has provided a range of services that have ensured a sound management of IP within the agency, in addition to providing additional support for the implementation of the contribution component of the STDP.

As a result of these achievements, the CSA has enhanced its understanding of the range and nature of the challenges associated with potential missions, and the promising avenues that exist to address these challenges. The CSA is also more readily engaged in enhancing the capability of the Canadian space industry, making it better positioned to engage in projects led by the CSA or by other agencies or entities, and to undertake commercial endeavours as applicable.

The evaluation also indicates that the SSP is being delivered efficiently. The STDP group has successfully carried out a broader mandate, particularly as it relates to the implementation of the contribution components, while keeping the level of operational resources fairly consistent. The program appears to be sufficiently flexible to accommodate unforeseen developments, although it may be worth exploring whether further flexibility could be accommodated to ensure that RFPs or AOs remain focussed on their original purpose, namely to explore a range of priority technologies based on planning activities. The IPMTT group has also managed an increase in its overall level of activities, including both an increase in ongoing services, as well as initiatives specific to identified missions of the CSA or new IP requirements.

1. Introduction

This document constitutes the final report of the evaluation of the Enabling Technology Development Sub-Sub-Program (hereafter the Sub-Sub-Program or the SSP). More specifically, the evaluation focusses on two components of the SSP: the Space Technology Development Program (STDP) and the services offered by the Intellectual Property Management and Technology Transfer (IPMTT) group. It covers a five-year period, from April 1, 2010 to March 31, 2015, and explores various dimensions of program relevance, design, and performance.

The evaluation is a requirement of the CSA's five-year evaluation plan; it was conducted in accordance with the Treasury Board Secretariat's Policy on Evaluation (2009), requiring all federal programs to be evaluated every five years.

The evaluation was conducted by PRA Inc. on behalf of the CSA Audit and Evaluation Directorate, between September 2015 and March 2016.

The contribution and collaboration of many individuals have made this evaluation possible. We wish to thank all of those who participated in data collection, provided information, and responded to inquiries.

2. Background

This section of the report includes a brief description of the SSP. It covers its key components, governance model, resource allocation, and expected outcomes. The purpose of this section is to provide sufficient contextual information to adequately assess the evaluation findings presented in the remaining sections of the report.

2.1 Program profile

2.1.1 Policy context

In order for the CSA to pursue its fundamental goals related to "the peaceful use and development of space,"Footnote 1 it must nurture, sustain, and grow the required technological knowledge within both the agency and the Canadian space sector. More than ever, space programming and ventures (both public and private) are expected to respond to an increasing reliance among nations on the information they provide. Every Canadian, every day, benefits from the various applications of space systems.Footnote 2 Whether it relates to the delivery of emergency, health, and other public services; personal and commercial communications; Global Positioning System (GPS) enabled tools; the management of agricultural activities; the commercialization of natural resources; or the monitoring of environmental trends, space data and applications rest at the centre of processes that support such activities.Footnote 3 Yet, there is simply no harsher environment in which technology operates than space. In addition to the many challenges that space represents in itself, countless orbital debris that have accumulated over the years add significant risks that may jeopardize access to critical space data.Footnote 4

In this context, various space agencies around the world have put technology development at the forefront of their mandate and operational priorities.Footnote 5 The CSA makes no exception. Through innovative approaches, new space applications can expand the range and depth of available data, in addition to enhancing the management and long-term sustainability of activities in space.

At the time of the evaluation, and as illustrated in Figure 1, CSA's programming activities covered three key areas: space data, space exploration, and Canadian space capacity. In the latter case, capability-buildingFootnote 6 was expected to be achieved by enabling technology development and by providing access to international markets. Considering Canada's positioning in the field of space activities, access to international markets has historically been linked to technology development, which explains the direct relationship between these two activity areas.Footnote 7

Figure 1: Strategic Outcome and Program Alignment Architecture (2015-2016)

Strategic Outcome 1

Canada's exploration of space, provision of space services and development of its space capacity meet the nation's needs for scientific knowledge, innovation and information.

Strategic Outcome and Program Alignment Architecture (2015-2016)
Text description of image for Figure 1
  • Programs
    • 1.1 Space Data, Information and Services
    • 1.2 Space Exploration
    • 1.3 Future Canadian Space Capacity
    • 1.4 Internal Services
  • Sub-Programs
    • 1.3.1 Space Expertise and Proficiency
    • 1.3.2 Space Innovation and Market Access
    • 1.3.3 Qualifying and Testing Services
  • Sub-Sub-Programs
    • 1.3.2.1 International Market Access
    • 1.3.2.2 Enabling Technology Development
      • STDP
      • IPMTT

As noted in introduction, this evaluation explores more specifically two groups of activities that the CSA is undertaking to support technology development — namely the implementation of the STDP and the management of intellectual property (IP) rights through the IPMTT group. It must be emphasized that the CSA does undertake other types of activities that support technology development, including those technologies that relate to confirmed missions in which the CSA participates. These activities are excluded from the scope of this evaluation, as they are covered under other programming.

2.1.2 Overview of the STDPFootnote 8

The STDP is an ongoing R&D program of the CSA that provides financial support through contracts and contribution agreementsFootnote 9 to entities that are selected, based on their capacity to develop specific space technologies. The two fundamental and interrelated goals of the STDP are to the following:

  • formulate, implement, and manage R&D contracts related to the development of mission-enabling and generic technologies to support the future needs of the Canadian Space Program;
  • support industrial capability-building through the development of new products and services, processes, and know-howFootnote 10.
2.1.2.1 Research contracts

The CSA issues research contracts to selected entities for the development of technologies (both generic and mission-enabling technologies) that respond to specific needs that it has identified. As such, these contracts are expected to support potential missions being considered by the CSA.

The process that the CSA uses to allocate research contracts includes a number of key steps that are illustrated in Figure 2 (next page), and can be summarized as follows:

Figure 2: Overview of the STDP research contract allocation process
Overview of the STDP research contract allocation process
Text description of image for Figure 2
  • Three core sectors identify technology development needs (Space Utilization; Space Science and Technology; Space Exploration)
  • Technology Development Requests (TDRs)
  • Technology Planning Working Group (TPWG) consolidate the list of priorities
  • STDP establishes cut-off point-based on available funding
  • Steering Committee reviews and approves the final list of proposed contracts
  • STDP launches the Request for Proposals process
  • Public Services and Procurement Canada (PSPC) provides support to the RFP process
  • STDP manages research contracts
  • Based on their respective mission plans or mission roadmaps, each of the Agency's three program sectors (Space Utilization, Space Exploration, and Space Science and Technology) identifies what it considers to be its current technology development needs. These needs are documented through formal Technology Development Requests (TDRs). The value of each proposed contract in a TDR will typically vary between $100,000 and $500,000.
  • TDRs are submitted to the TPWG, which is tasked with the responsibility of consolidating the three lists into one ranked list of potential research contracts.
  • Based on available funding, and in consultation with the three program sectors, the STDP group recommends the list of research contracts that could be considered for approval. A Steering Committee, composed of the three sectors' DGs, reviews and approves the final list of potential research contracts.
  • In order for the research contracts to be established, the STDP group works directly with PSPC, formerly Public Works and Government Services Canada). The role of PSPC is procedural in nature. It issues RFPs, receives all bids submitted in response to RFPs, and monitors the overall process to ensure that all federal rules related to procurement contracts are respected. The actual assessment of the bids is done by the STDP group, with the support of technical or scientific authorities as required, based on the nature of the proposed assignments.
  • Once the STDP group has signed a contract with an entity (normally a private firm or a university), a member of the group is assigned to act as program authority and is responsible for the ongoing management of the contract up until its completion. This person collaborates with the representatives of the core sector that initially came up with the TDR.
2.1.2.2 Contributions

Since 2012, the CSA offers non-repayable contributions to selected entities in order to support the development of innovative space technologies. While, by their very nature, these technologies are of interest to the CSA, they are not intended to support potential missions of direct interest to the CSA. Their primary goal is to support the development of Canadian industrial capabilities in the area of space technologies for the purpose of increasing the commercial potential of Canadian space companies.

The process that the CSA uses to issue contribution agreements includes steps that are illustrated in Figure 3 and can be summarized as follows:

Figure 3: Overview of the STDP contribution allocation process
Overview of the STDP contribution allocation process
Text description of image for Figure 3
  • The STDP, in collaboration with the industry representatives and other stakeholders as required, identifies eligible technology themes
  • STDP launches the Announcement of Opportunities (AO) process
  • G&C Center of Expertise supports the A process
  • STDP manages Contribution agreements
  • The CSA works directly with Canadian space industry representatives (including, for instance, the Aerospace Industries Association of Canada) and other stakeholders as required, to determine the broad themes that could be covered by STDP contributions. More specifically, the STDP has been using the areas identified in Canada's Space Policy FrameworkFootnote 11 as a basis, to which other related themes have been added as required to reflect the ongoing needs of the Canadian space industry in terms of technology development.
  • In order to solicit applications from industrial partners, the CSA issues AOs on its website, through its Class Grant and Contribution Program to Support Research, Awareness and Learning in Space Science and Technology (Class G&C Program). Each AO will specify the maximum funding available for each agreement. For instance, the AO issued in 2014 for the STDP included two categories of agreements, one offering contributions of up to $200,000 and one offering contributions of up to $600,000. In all cases, funding is provided for terms of up to 24 months. Also, government funding of any source (federal, provincial, or municipal) can only cover 75% of the total eligible costs for any contributions funded through an AO.
  • In order to establish and manage contribution agreements, the STDP group works directly with the CSA Grant and Contribution Centre of Expertise (G&C COE). Much like PSPC does for research contracts, the G&C COE ensures that all CSA rules related to contributions agreements are respected, and provides organizational support during each phase of the contribution process.
  • Once a contribution agreement has been established, a member of the STDP group is assigned to act as program authority and is responsible for the ongoing management of the agreement up until its completion.
2.1.2.3 Financial and human resources

The CSA invested close to $48 million in the STDP during the period covered by the evaluation. Table 1 presents the distribution of this funding.

Table 1: Actual STDP spending, 2010-2011 to 2014-2015 ($000's)Footnote 12

  2010-11 2011-12 2012-13 2013-14 2014-15 Total
Programming allocations
Contract allocations 11,005 4,171 6,844 8,952 8,017 38,990
Contribution allocationsFootnote 13 n/a n/a 90 589 1,492 2,171
Total programming 11,005 4,171 6,934 9,542 9,509 41,161
STDP management of research contracts and contribution agreements
SalariesFootnote 14 1,073 1,097 1,316 1,083 1,357 5,925
O&M excluding contractsFootnote 15 131 235 72 58 230 727
Total STDP management 1,204 1,332 1,389 1,141 1,587 6,652
Total program spending 12,209 5,503 8,323 10,683 11,096 47,813

Source: CSA financial data, February 2016

The most significant variations in the table come from the level of funding for research contracts that has been allocated during each fiscal year. Following a decline in 2011-12, this funding has increased since 2012-13 and is expected to grow further, bringing the total operations and maintenance (O&M) costs (including research contract) of the STDP to $13.1 million by 2016-17. Also, contributions were first issued in 2012, on a pilot basis, before being fully implemented in 2014. Just like research contracts, the level of funding for contributions has been increasing, reaching $8.2 million in 2015-16, and $7 million for the following years. Looking forward, and starting in 2015-16, the planned budget for the STDP will be at least $20.9 million annually.

As should be expected, these important variations in annual funding allocations for contracts and contributions have, in turn, triggered significant shifts in the ratio of overhead costs to funding allocations. For instance, while the ratio of salaries to funding allocations stood at 14.4% during the period covered by the evaluation, it is expected to fall down to 6.4% starting in 2015-16. This will, in fact, bring the ratio back to where it stood during the period covered by the previous STDP evaluation (2002-03 to 2007-08).Footnote 16

The CSA allocated an average of 5.3 full-time equivalents (FTEs) for the direct administration of the program during the period covered by the evaluation. In addition, an average of 5.9 FTEs operating in other sectors of the CSA supported the implementation of STDP activities (as technical authorities for instance).

2.1.2.4 Governance structure

The Director General of Space Science and Technology is accountable for the implementation of the Future Canadian Space Capacity program (as described in Figure 1), which includes the STDP. The Technology Development Management directorate is responsible for the daily operations of the program.

2.1.2.5 Program theory

Appendix A of this report includes a detailed description of the STDP's logic model. For the purpose of this program overview, the logic model's key outputs and outcomes are summarized in this sub-section.

Through the implementation of research contracts related to generic and mission-enabling technologies and contribution agreements aimed at enhancing industrial capabilities, the STDP is expected to contribute to the following outcomes:

Immediate outcome

The understanding of technical challenges relating to space technologies and their solutions is increased.

Intermediate outcomes

Technical uncertainties and unknowns linked to space missions or activities are reduced.

The number of innovative generic technologies that have the potential of contributing to space-related endeavours is increased.

The technological capacity of the Canadian space sector is increased or maintained.

Ultimate outcome

The Canadian space sector's ability to respond to Canada's current and future space objectives, as well is its ability to be competitive in the global market, are enhanced.

2.1.3 Overview of the IPMTT groupFootnote 17

The IPMTT group provides expertise on a range of issues related to IP management, technology transfers, and technology commercialization. In doing so, it works collaboratively with CSA's three program sectors (Space Utilization, Space Exploration, and Space Science and Technology); CSA corporate functions (Legal Services, Information Management, and Finances); other federal departments, such as PSPC; and the Canadian space industry and academia.

The fundamental goal of the IPMTT group is to facilitate the use and sharing of space technologies and products for space applications and, eventually, terrestrial applications. This ensures that Canadian-funded activities related to space technologies that generate IP rights are optimized for the benefit of the CSA and of Canada as a whole.

2.1.3.1 Range of services offered

The range of activities that the IPMTT group undertakes to fulfill its mandate includes the following:

  • managing all the IP assets that are created through CSA activities, including licences, copyrights, trademarks, official marks, industrial secrets, and patents;
  • supporting and facilitating the transfers of Crown-owned technologies to third parties to allow for the commercial use of these technologies;
  • providing opinions and advice, through a single window process, on any IP-related questions, including matters relating to copyright law or IP clauses to be included in research contracts signed by the CSA;
  • offering training activities on IP-related issues (online, in-person, or customized), and providing general information aimed at raising awareness of IP issues among all CSA employees.
2.1.3.2 Financial and human resources

During the period covered by the evaluation, the CSA invested an average of $728,387 annually to support the work of the IPMTT group. Table 2 presents the distribution of this funding between salaries and operations and maintenance.

Table 2: Actual IPMTT spending, 2010-2011 to 2014-2015 ($)

  2010-11 2011-12 2012-13 2013-14 2014-15 Total
SalariesFootnote 18 289,480 328,828 332,893 354,090 371,963 1,677,254
O&M 458,570 480,084 335,620 305,845 384,564 1,964,683
Total spending 748,050 808,912 668,513 659,935 756,527 3,641,937

Source: CSA financial data, February 2016.

During the same period, the CSA allocated an average of 3.75 FTEs to the operations of the IPMTT group.

2.1.3.3 Program theory

A detailed description of the IPMTT group's logic model is included in Appendix B of this report. For the purpose of this overview, the logic model's outputs and outcomes are summarized in this sub-section.

Through the advice it provides, the training activities it undertakes, and the management function it assumes, the IPMTT group is expected to contribute to the following outcomes:

Immediate outcomes

Access rights, for government and non-government stakeholders, to space technologies that are supported by the CSA are secured.

Appropriate protection strategies to support government and non-government projects and endeavours involving Crown-owned space technologies are established.

Intermediate outcome

Crown intangible assets are managed to reduce risks associated with IP matters and to maintain the innovation chain continuum.

Ultimate outcome

The circulation of space technologies and products for space and terrestrial applications benefitting Canadians is facilitated.

2.2 Prior evaluation of the program

It is the first time that the SSP as a whole has been formally evaluated. Back in 2011, the CSA published an evaluation that focussed specifically on the STDP, which covered a six-year period, from 2002-03 to 2007-08.Footnote 19 The set of activities undertaken by the IPMTT group has never been evaluated in the past.

The 2011 evaluation of the STDP addressed the issues of relevance, design and delivery, program success, and cost-effectiveness and alternative. While that evaluation concluded that the program was relevant and performing generally well, the report identified areas for improvement, including two that appear particularly important in the context of this evaluation. First, the 2011 evaluation noted that, in the absence of a mechanism to issue contributions, it was challenging for the CSA to have a clear direction on how it intended to specifically enhance the industrial capability of the Canadian space sector. Secondly, the evaluation noted that the CSA was not in a position to offer a clearly articulated mission plan that could inform decisions made through the STDP. These two issues are explored once more in the current evaluation.

The 2011 evaluation of the STDP included the following six recommendations, all of which have been implemented:Footnote 20

  • Greater clarity regarding the objectives of the contracting program is needed.
  • CSA needs to clearly map all proposed missions and related R&D technology requirements.
  • STDP personnel and Public Works and Government Services Canada (now PSPC) need to clearly map the procurement and contracting process, roles of each party, service standards, and their requirements.
  • There should be further study on the possibility of introducing a two-step application process: 1) letter of interest detailing the technology, and 2) complete proposal if requested.
  • CSA needs to improve communications with industry and stakeholders through announcements or workshops, ahead of the release of its RFPs.
  • There should be a formal strategy to communicate project results to interested CSA parties.

3. Evaluation approach and methods

This section of the report provides a brief description of the methodology used to conduct the evaluation of the SSP. It clarifies the purpose and scope of the evaluation, describes the key evaluation issues being addressed, and the methods used to gather evaluation findings. It also identifies the limitations that the evaluation faced, along with the strategies used to mitigate these limitations.

3.1 Purpose and scope

This report fulfills the commitment included in the CSA's Departmental Evaluation Plan (2015-16 to 2019-20) to conduct the evaluation of the Enabling Technology Development SSP.Footnote 21 It covers a five-year period, from 2010-11 to 2014-15.

The core issues included in the Directive on the Evaluation Function and relating to the relevance and performance of the SSP are covered in this evaluation. In addition, questions on the design of the SSP have been added to reflect changes to the delivery of some of its components. These questions are as follows:

Relevance

  • Is the SSP (the STDP and the IPMTT groups) aligned with the federal government's and the CSA's goals and priorities related to space technology development?
  • To what extent do activities undertaken through the SSP (the STDP and the IPMTT groups) adequately reflect the overall distribution of the roles and responsibilities related to space technology development?

Program design

  • How efficient is the STDP's program structure in supporting the development of space technology in Canada?
  • How efficient is the IPMTT group's service structure in supporting the Crown's intangible assets (as it pertains to the development of space technology in Canada)?
  • Is the SSP supported through an adequate performance measurement strategy?

Performance (effectiveness)

  • To what extent has the SSP achieved its immediate outcomes?
  • To what extent has the SSP achieved its intermediate outcomes?
  • Have any unexpected outcomes resulted from the activities undertaken through the SSP?

Performance (efficiency and economy)

  • What measures have the CSA implemented to optimize the use of the SSP's resources (the STDP and the IPMTT groups)?

3.2 Methods

3.2.1.1 Document and data review

The first step in the document review consisted of a review of documents related to the STDP and to the IPMTT groups, to assist in addressing evaluation questions related to the continued relevance and performance of these programs. This portion of the document review only considered documentation generated by the program. As such, it excluded evidence from the various performance tracking systems (which was included in the database review described below). In particular, the types of documents for review included the following:

  • legislation pertaining to space technology development
  • space technology-related policies and strategies
  • CSA planning documents
  • program descriptions, objectives, and requirements
  • other departmental documents (e.g., research documents, performance reports, presentations, and background documents)
  • previous evaluations and audits

The second step consisted of reviewing the administrative data, which provided quantitative information related to activities carried out through the STDP and by the IPMTT groups. This information addressed many of the evaluation questions covered by the evaluation.

First, the review gathered primary information on all R&D projects funded by the STDP (both through contracts and contribution agreements).Footnote 22 The information included, in particular, the number of contracts and contribution agreements signed, the level of funding allocated, the internal clients served, and information on timeframes associated with each R&D project.

Second, the review included a more in-depth analysis of data relating to a selected number of R&D projects funded through the STDP. The goal was to select a range of representative projects, and proceed with a systemic gathering of information that could build on the primary information described above. As such, the additional information addressed the actual outputs and outcomes that have been achieved by the projects, and gathered other relevant information that could speak to the efficiency of the program design and its performance.

The sampling of R&D projects was purposeful in order to represent a range of variation in dimensions of interest (e.g., contracts/contributions, larger/smaller projects, mission-enabling/generic technologies contracts, projects requested by CSA's Space Utilization/Space Exploration Directorates). The projects were identified by the evaluation team, in collaboration with the STDP managers. The number of selected projects was determined based on the quality, accessibility, and extent of the information available on each project.

Finally, the review gathered information on activities and services provided by the IPMTT group. The goal was to use this information to illustrate the range of activities undertaken by the group, as well as the target groups that are expected to benefit from these activities. In addition, it provided contextual information that supported the analysis of the information gathered through other lines of evidence.

3.2.1.2 Key informant interviews

Key informant interviews contributed to the in-depth understanding of the STDP and the IPMTT groups, including results achieved and challenges faced by these program components. These interviews also corroborated, explained, or further elaborated on findings from other data sources and provided important input into whether outcomes have or have not been achieved, and why they have or have not been achieved.

A total of 19 interviews of approximately one hour were conducted with 31 key informants from eight different stakeholder groups. Each group is described in the following table.

Table 3: Distribution of interviews conducted as part of the evaluation

Key informant groups # of interviews # of individuals interviewed
CSA senior management 5 5
STDP managers and staff 1 7
IPMTT managers and staff 1 2
CSA Space Exploration clients 1 3
CSA Space Utilization clients 1 3
CSA G&C Centre of Expertise 1 2
Public Services and Procurement Canada 1 1
Industry and academia 8 8
Total 19 31

3.2.1.3 Case studies

Case studies on three specific R&D projects funded through the STDP provided more in-depth information on successes, challenges, and best practices, as well as proven and anticipated impacts of the SSP. These case studies also highlighted or confirmed findings from other lines of evidence.

Three projects funded through the STDP were selected, including both research contracts and contribution agreements. Just like any R&D project funded through the STDP, they also included issues related more specifically to IP.

Each case study included up to three interviews with stakeholders associated with the funded projects, including external stakeholders (e.g., private sector organizations, post-secondary institutions), internal CSA stakeholders (personnel responsible for identifying future technology requirements), and representatives from the STDP and IPMTT groups. In total, seven interviews were conducted, involving 11 individuals.

Case studies also involved a review of project-level documents and data maintained by the CSA on each of the funded projects, as well as publicly available information (e.g., websites, online documents) and a more in-depth analysis of the files/data maintained by the CSA for each of the projects.

3.3 Limitations

A number of data limitations related to the evaluation of the SSP had to be addressed.

Performance measurement strategy development

The performance measurement strategies for both the STDP and the IPMTT groups were developed and approved at the onset of the evaluation. As a result, only limited performance information had been collected on the basis of the performance indicators included in the strategy. Regardless, these strategies provided, at a minimum, an updated description of both the STDP and the IPMTT groups and provided indications of how they intend to collect data and the nature of the data collected.

Changes to the Sub-Sub-Program

The SSP has undergone substantial changes since 2010. During the evaluation period, the SSP has lost two components (Advanced Technology Planning and Technology Management), and has acquired one (the IPMTT group). As a result, the evaluation is addressing two programs/services with distinct objectives and activities. Whenever possible, links between the STDP and the IPMTT components were established in the evaluation methodology and findings.

Scope of key informant interviews

While all key stakeholder groups were consulted as part of the evaluation, these findings remain qualitative in nature. As such, interview findings are not statistically representative and were considered along with other data collection activities through an appropriate triangulation process.

Limited scope of the consultations held in relation to IPMTT group activities

The range of activities undertaken by the IPMTT group reaches beyond those undertaken in support of the STDP. While some of the data collected cover this broader range, the majority of the evaluation findings on the IPMTT group are, in fact, closely aligned with STDP activities. For instance, no interviews were conducted with clients of the IPMTT group beyond those who are also involved with the STDP. This is reflected in the range of analysis contained in this report.

Limitations of UNITAS data

The CSA's database, UNITAS, does not systematically generate information on all aspects of funded R&D projects. For instance, for projects reviewed as part of this evaluation, shifts in TRLs, total projects expenditures, or contract details were available for most, but not all, contracts. This created challenges when attempting to produce aggregate information on the overall projects that have received funding under the STDP. As a result, the evaluation team worked closely with the STDP group to determine the nature and range of information and the data available for the purpose of the evaluation. This underlined the importance of conducting interviews and case studies, which provided more in-depth information on projects and activities funded under the program.

CSA client survey

Program documentation indicated that a client survey was conducted in 2014 to assess the relevance, outcomes, and performance of the STDP. The survey targeted internal CSA clients of STDP contracts awarded between 2009 and 2012 and, as such, was not meant to gather opinions from external stakeholders, such as industry stakeholders. As part of this evaluation, these gaps were addressed through other lines of evidence, such as the case studies, allowing the evaluation to gather the perspectives of the industry. A second client survey was undertaken in 2015, during the time this evaluation was conducted. Preliminary results of this client survey were used to illustrate the program's relevance and performance.

STDP non-funded recipients

Due to budget and scheduling limitations, it was not possible for the evaluation team to gather the views of non-funded recipients (through contributions) on the need for the STDP, and the extent to which R&D projects submitted by these non-funded recipients were able to proceed without STDP funding.

Assessment of ultimate outcomes

Budget and scheduling limitations also had an impact on the range of issues that could be addressed through this evaluation. While the achievement of immediate and intermediate outcomes is addressed in this report, the evaluation did not explore the extent to which the ultimate outcomes of both components of the SSP were achieved. The range and nature of the data required to adequately measure the SSP's ultimate outcomes, combined with the different types of activities undertaken by the STDP and the IPMTT groups, would have required a significantly expanded methodology.

Most interviewees have a vested interest in the SSP

This limitation was mitigated by requiring interviewees to explain their perspectives and provide examples where appropriate. In terms of the overall report, the findings from the key informant interviews were triangulated with findings from other data sources (document review, administrative data, financial data, and case studies).

Space technology development process

The nature of space is such that it takes many years to fully develop technologies. As the development of space technologies requires long lead times, it may be too early to assess the mid- and long-term benefits of the technologies developed through the STDP, in addition to IP issues that may arise later in the innovation chain.

4. Results

This section of the report describes the evaluation findings related to the SSP. The information is based on findings that emerged from all of the lines of evidence. Unless otherwise noted, when opinions are reported, these are the opinions of the stakeholders consulted, and not those of the evaluators.

4.1 Relevance

For the purpose of this evaluation, the relevance of the SSP is determined by the extent to which it is appropriately aligned with the goals and priorities of the CSA, and of the Government of Canada more generally. It also explores the extent to which the SSP is adequately aligned with the distribution of roles and responsibilities, when it comes to space technology development.

4.1.1 Alignment with federal government goals and priorities

The future of Canada's involvement in space activities has been addressed through a number of key policy initiatives that emerged during the period covered by this evaluation. While several dimensions of space activities have been covered, the actual development of space technology has often played a predominant role in these initiatives.

4.1.1.1 R&D as economic driver

In 2010, as the world economy was coping with the aftermath of the financial crisis, the federal government announced significant investments in space activities as part of its economic action plan, including close to $400 million in support of the RADARSAT Constellation Mission (RCM), and $110 million toward other space initiatives, such as the development of the Next Generation Canadarm.Footnote 23 Also part of the 2010 federal budget was the launch of an expert panel mandated "to enhance federal programming in support of a more innovative economy," particularly through R&D initiatives.Footnote 24 In its report tabled in the fall of 2011, the panel noted that the CSA was a leading federal agency in engaging the industry in R&D activities, and recommended that the federal government provide long-term opportunities for Canadian business to address pre-commercialization gaps through R&D activities.Footnote 25

4.1.1.2 The Emerson Report

As the broader review of R&D initiatives was pursuing its course, the federal government also announced, in its 2011 budget, "a comprehensive review of all policies and programs related to the aerospace/space industry to develop a federal policy framework to maximize the competitiveness of this export-oriented sector and the resulting benefits to Canadians."Footnote 26 Framed as part of federal initiatives to foster job growth, the review was expected to involve key stakeholders, particularly the aeronautic and space industry. In what came to be known as the Emerson Report, the review contextualized the challenges face by Canada when it comes to building a strong and sustainable space industry, by highlighting the crucial need to maintain a high level of competitiveness in foreign markets:

"Historically, space-related activity has largely been led by governments. Motivated partly by prestige, partly by curiosity, and partly by the desire to support provision of public services, governments have borne much of the cost and risk of space exploration and activity. Where market economies exist, governments have done so in partnership with companies that have received contracts to design and manufacture space assets for public as well as private use. In Canada, the result has been the creation of a $3.4 billion space industry that employs 8,000 workers across the country, derives 80 per cent of its revenue from satellite communications, and generates half of its revenue from sales abroad, making it one of the most export-oriented space sectors in the world."Footnote 27

Focussing on current challenges faced by the space industry, the Emerson Report called for a clearer vision as to the goals that Canada wishes to achieve in relation to space activities and how it intends to meet them. The report pointed to an:

"inadequate clarity of purpose with respect to Canada's space program and its role in providing services and advancing national priorities. This lack of focus appears to go back at least a decade and has been manifested in weak planning, unstable budgets, and confusion about the respective roles of the CSA and those government departments that are major space users. In a sector whose undertakings are, by definition, long-term, expensive, and complex, it is especially important to have concrete goals, predictable funding, and orderly implementation."Footnote 28

Two of the recommendations from the Emerson Report are particularly relevant for the purpose of this evaluation. First, the report urged the government to "explicitly recognize the importance of space technologies" by implementing a government-wide approach to policy and programming development (as opposed to an agency-wide approach). Secondly, it recommended an immediate increase to support technology development: "Total funding for the Canadian Space Agency's technology development programs [should] be raised by $10 million per year for each of the next three years, and … maintained at that level."Footnote 29

The report did emphasize that space technology development is a risky and long-term endeavour, but that since its potential benefit to Canadian society is substantial, the federal government must directly engage in sharing its associated risks.Footnote 30

4.1.1.3 Canada's Space Policy Framework

Partly in response to the Emerson Report, in February 2014, the federal government unveiled Canada's Space Policy Framework.Footnote 31 One of the principles guiding this new framework is the predominant role that the space industry is expected to play:

"As space yields ever more commercial opportunities, the government will focus on supporting the domestic space industry in the innovation required to bring to market cutting-edge technologies that meet national interests; and utilizing industry where industry has greater capacity, knowledge and skill, or when it can be more efficient and cost-effective."

As it relates to space technology development, the framework paves the way for a government-wide approach that would directly involve the CSA and other federal partners. To this end, it contains three interrelated investment strategies:

  • increasing support for technology development, especially in areas of proven strength, such as robotics, optics, satellite communications, and space-based radar, as well as in areas of emerging expertise;
  • coordinating with the granting councils and foundations to ensure that space research resources are leveraged and that space research itself figures prominently in their mandates;
  • leveraging existing expertise and programs at the National Research Council, Defence Research and Development Canada, Communications Research Centre Canada, and the Strategic Aerospace and Defence Initiative — including the newly announced Technology Demonstration Program — to better support the space industryFootnote 32.
4.1.1.4 Seizing Canada's Moment

Following the release of the new space policy framework, in December 2014, the federal government published an updated science, technology, and innovation strategy entitled Seizing Canada's Moment: Moving Forward in Science, Technology and Innovation.Footnote 33 In addition to echoing the commitment from the space policy framework to further invest in space technology development (among other technologies), the new strategy emphasized the pivotal role that IP rights occupy in the innovation chain:

"Canada's innovation support programs need to help Canadian entrepreneurs develop the skills and knowledge they need to benefit from the commercial advantages of intellectual property. This need is particularly acute for small and medium-sized enterprises, many of which enter new markets without adequately having thought through their intellectual property strategy."Footnote 34

As such, the government committed to not only update the legislative and regulatory frameworks relating to IP rights, but also to provide direct assistance to industry in order to enhance its capacity to protect and appropriately manage their intellectual property.

4.1.1.5 The alignment of the SSP

In itself, the context described in the preceding sub-sections speaks to the relevance of activities undertaken through the SSP. At the time of the evaluation, the STDP remained the primary tool available to the federal government to support space technology development activities. While other federal programs — such as the Strategic Aerospace and Defence Initiative (SADI) or the Technology Demonstration Program (TDP) — may provide some support to space technology development, interviews conducted as part of this evaluation indicate that none of these alternatives provide the range and depth of support that the STDP is currently offering. It was further noted that the nature of space technology is significantly different from aerospace technology, insofar as the latter can typically be commercialized within a shorter timeframe, leading to more immediate commercial returns. As such, repayable contributions, such as those offered by SADI, find less application in space-related technology development.

While the predominant role played by the STDP in space technology development substantiates its relevance, it also signals — as noted during interviews — that the goal expressed in the space policy framework to have a government-wide approach to support space technology development has yet to be fully achieved. This, however, can arguably be considered a shortcoming of the framework implementation, and not so much of the SSP or the STDP.

Since building the capability of the space industry to respond to the needs of the CSA and be competitive internationally is a stated goal of the federal government, the inclusion of a contribution component as part of the STDP (starting in 2012) also contributes to its relevance. Evaluation findings indicate that the STDP has historically pursued capability-building activities; however, having a formal mechanism that is specifically designed for this purpose is seen to be more appropriate. This also responds to one of the key recommendations of the last STDP evaluation, conducted in 2011.

It is important to emphasize that — with an updated science, technology, and innovation strategy in place that emphasizes the need for sound IP rights management — the SSP is well positioned to maximize the application of space technology for which the CSA has provided financial support or that has been internally created in the past (Crown-owned technologies). The range of activities undertaken by the IPMTT group coincides with the scope of IP management expected of all federal departments and agencies, as identified in the strategy as well as in applicable legislation. In addition to providing a single window approach to all IP-related issues within the agency, the range of activities undertaken by the IPMTT group also reach external partners, particularly those that receive financial support (through the STDP or other CSA programs) to develop generic, mission-enabling, or mission-confirmed technologies. This comprehensive approach to IP management is particularly important for the period covered by the evaluation, as internal technology development largely ceased in 2012, which essentially reduced the creation of IP rights within the CSA.Footnote 35 As already noted, it is also during this period that the CSA has implemented the contribution component of the STDP, which triggers new requirements for the IPMTT group, particularly as it relates to assessing the commercial potential of R&D projects seeking financial support through contributions.

4.1.2 Alignment of roles and responsibilities

4.1.2.1 The role of the private sector

Considering the scope of space activities and the complexity of their associated technologies, it will come as no surprise that the CSA, just like all other space agencies around the world, must turn to the private sector to conceptualize and build almost all technologies that are eventually included in its missions. In normal market circumstances, governments would readily engage in procurement activities, assuming that private businesses would promptly occupy the market share that systematically results from these demands. But such normal market circumstances hardly apply to space.Footnote 36 As noted throughout interviews, space missions typically require narrowly defined technologies that may well be applied on a single occasion, and that may offer very little, if any, potential for broader commercialization.

Under which circumstances, then, could space technology developers operate in a sustainable manner? And what would be the consequences if the CSA was to find itself incapable of fulfilling its space technology requirements? Evaluation findings confirm that, in the absence of deliberate strategies to support and grow the Canadian space sector, the CSA would most likely face serious limitations in its ability to pursue its space agenda.

In the context of space activities, supporting the private sectors entails at least two complementary dimensions. First, the government must directly support efforts undertaken by the space sector to develop promising technologies that may not find direct applications in the short term, but that may pave the way for long-term benefits. Currently, this is largely done through contributions provided by the STDP. Second, the government must align its procurement strategy in such a way as to favour activities undertaken by the private sector (as opposed to activities undertaken internally) when appropriate. This is largely achieved through research contracts under the STDP. This two-level approach is directly reflected in Canada's Space Policy Framework:

"As space yields ever more commercial opportunities, the government will focus on supporting the domestic space industry in the innovation required to bring to market cutting-edge technologies that meet national interests; and utilizing industry where industry has greater capacity, knowledge and skill, or when it can be more efficient and cost-effective."Footnote 37

This view also reflects findings from the Emerson Report, which emphasized that the role of the government is to support "research and development (R&D) that might take years to produce marketable results but has the potential to generate substantial benefit to the public good, in part through risk sharing," as well as to make "procurement decisions that strengthen domestic industries, and therefore the national economy, while respecting international trade rules and acquiring the best product for a reasonable cost."Footnote 38

Arguably, this ultimately aligns with the fundamental mission of the CSA, as stated in the Canadian Space Agency Act, which is to ensure that "space science and technology provide social and economic benefits for Canadians."Footnote 39

4.1.2.2 The role of the CSA

Beyond the rationale for enhancing the capacity of the space sector to conduct R&D activities, the evaluation also explored the extent to which the CSA should directly engage in space technology development. At a minimum, and as already noted, the SSP supports the ongoing management of IP rights related to space technology, and manages research contracts and contribution agreements through the STDP. But should the CSA conduct internal R&D activities as well?

A working group established in support of the 2012 aerospace review argued in favour of redirecting some of the R&D activities done internally by the CSA toward the space industry:

"With respect to R&D, the CSA has over the years developed an internal R&D capacity as part of its legitimate need to be aware of the latest technology trends in the space field. However, in some cases, industry feels that the CSA is conducting R&D that would better serve the nation if it were done in industry and academia. Doing this R&D extramurally would enhance the competitiveness of the industry and would support a broader base of space science activity in our universities."Footnote 40

It is important to note that the view expressed at the time was about rebalancing R&D activities between what was done internally and what was being contracted out. The working group did emphasize that "the CSA needs an internal R&D capacity in order to be an 'intelligent' manager of the Canadian Space Program."Footnote 41

Evaluation findings indicate that instead of rebalancing these activities, the CSA essentially ended its internal R&D activities related to space technology development, as part of a strategic review process. While some marginal work may have proceeded during the evaluation period, there is no longer any systematic engagement in these types of R&D activities.

Evaluation findings indicate that the current approach to internal space technology development is problematic. The ongoing implementation of research contracts and contribution agreements under the STDP does require sufficient internal knowledge to be "an intelligent manager" of such activities. For all research contracts and contribution agreements, the STDP group does assign a technical authority that serves a critical monitoring and supporting role throughout the life of these R&D projects. At this point, the STDP group continues to access internal expertise that has been built over the years through means such as internal R&D activities. However, unless there are ways to replenish this pool of knowledge, it is unclear how the STDP group (or the CSA as a whole) will be in a position to adequately monitor the technical dimensions of its funded projects. This concern was expressed by both representatives from the CSA and from the space industry during interviews conducted as part of this evaluation.

4.2 Program design

In order to adequately meet the information requirements of the CSA in relation to the SSP, the evaluation explored a number of questions on the actual design of its two components, the STDP and the IPMTT groups. The primary goal was to better understand the extent to which the current delivery structure was adequately supporting the achievement of the SSP outcomes.

4.2.1 Efficiency of the STDP program structure

The analysis of the current STDP program delivery structure covers key milestones, namely the planning of space technology development, the release of RFPs and AOs, and the management of the resulting contracts or agreements. This sub-section also explores the process of consolidating space development activities within the CSA.

4.2.1.1 The planning process

The planning associated with RFPs and AOs under the STDP is an initial step that has significant ramifications for the ultimate success of the program. Among the wide range of potential technologies to explore, identifying the ones that are the most promising largely determines the extent to which the program can be in a position to contribute to achieving the core objectives of the CSA.

Evaluation findings leave little doubt that key factors well beyond the scope of the STDP have a direct impact on the success of the program's planning process. Among the predominant factors that have emerged during the course of the evaluation is the absence of a long-term plan establishing overall priorities for the CSA, in terms of space activities and missions, and their associated requirements for space technology development. This is a fairly long-standing issue that was raised in the Emerson Report (see sub-section 4.1.1 of this report); it was also noted in recent evaluations conducted by the CSA.Footnote 42 Individuals interviewed as part of this evaluation added that the current focus placed by the CSA on large projects, such as the International Space Station (ISS) and the RCM initiative, further limits the agency's ability to engage in this process of setting long-term space priorities.

As a result, the role of the STDP group has largely been to facilitate dialogue among the agency's key sectors (Space Utilization, Space Exploration, and Space Science and Technology) in order to identify their respective needs, and to come up with a coordinated approach toward investing space technology resources, through the research contract component of the STDP. In order to facilitate this process, and as described in sub-section 2.1.2, the CSA established the TPWG. Evaluation findings indicate that this structure has helped better coordinate the set of activities undertaken by each of the three sectors, and avoid the potential duplication of efforts by bringing all three sectors around the same table. These achievements, however, cannot be expected to compensate for the absence of a long-term space plan. At the time of the evaluation, the exact role that the TPWG was expected to play remained unclear, particularly in light of consolidation efforts that were underway and that are further discussed in sub-section 4.2.1.4 of this report.

4.2.1.2 Issuing RFPs and AOs

Issuing RFPs and AOs is an operational process that requires close collaborations between the STDP group and PSPC (for research contracts) and the G&C COE (for contribution agreements). Interviews conducted as part of this evaluation indicate that the working relationships between the STDP group and these two partners are progressing fairly well, as roles and responsibilities are clearly articulated and the three groups are in a position to carry out their respective mandate efficiently. Representatives from the CSA and the space industry who were interviewed also confirmed that the operational dimensions of the RFP and AO processes are well known and understood.

Beyond these operational considerations, evaluation findings do point to a number of challenges that relate to the planning and management of these RFPs and AOs. One concern that emerged more specifically from interviews with industry representatives relates to the actual planning of these RFPs and AOs. It appears challenging for some of the potential applicants to know when these processes will be launched and what their scope will be. Considering the time and effort required to appropriately respond, it is seen by some as a significant limitation of the current model. It was noted that other agencies, such as NASA and the ESA, have processes in place to engage the space industry early on, in order to build a common understanding of their respective requirements for space technology development. While these other examples may provide relevant insights, it must also be recognized that Canada is operating in an environment that is drastically different from that of the NASA or the ESA. Evaluation findings have not uncovered specific strategies for engaging the industry earlier in the process that would not create further delays in a process that is already quite lengthy.

Another challenge noted during interviews relates to the time required in order to obtain all approvals to issue a research contract or a contribution agreement. Several months may be needed to gather these approvals; this creates uncertainties among those who are waiting to confirm whether they must assign resources to these potential R&D projects. This is another example of where constraints that fall beyond the direct scope of the STDP can affect its delivery.

4.2.1.3 Managing contracts and contribution agreements

Once contracts or contribution agreements have been established, evaluation findings indicate that the STDP is well positioned to ensure their effective management. The assignment of a program authority, as well as a technical authority, ensures that adequate monitoring and support are provided throughout the life of the project. Representatives from the space industry who participated in interviews confirmed that the current process for managing contract and contribution agreements is efficient. STDP representatives were described as accessible and responsive, and technical authorities offer important insights that support the effective implementation of the funded projects.

The nature of reporting that is currently required has also been described as reasonable. While some external partners argued in favour of lessening the reporting requirements, they also acknowledged that public funds do require a certain level of reporting to ensure proper accountability.

Internal clients of the CSA who work with the STDP group are invited to complete a survey questionnaire, once their R&D projects are completed, to provide feedback on certain dimensions of their projects, including their level of satisfaction with the services they received from the STDP group. Data covering the period from 2009 to 2015, reviewed as part of this evaluation, point to a high level of satisfaction with the services provided by the STDP group. More than 90% of respondents were either somewhat or very satisfied with the quality of the services provided by STDP representatives.

4.2.1.4 The consolidation of space technology development

As noted in the introduction, the CSA funds the development of space technology through a number of programs, processes, and tools. Each tackles different dimensions of technology development, including the support to academia,Footnote 43 the expanded use of space data,Footnote 44 or the exploration of new opportunities to test technologies.Footnote 45 The CSA also funds the development of technologies related to confirmed missions, such as the RCM or the ISS, and collaborates with other entities on technology development initiatives.Footnote 46

It is in this context, and in order to avoid duplication, that the scope of the STDP is limited to the development of generic and mission-enabling technologies, in addition to the range of technological themes that are explored through contribution funding.

It is also in this context that the issue of whether the CSA should consolidate some of its technology development activities has been raised. For instance, the recent evaluation of the Advanced Exploration Technology Development (AETD) SSP recommended that the CSA "either create a clearer distinction between the Enabling Technology Development program and the AETD program or merge the two programs while ensuring that the planning and execution of technological, operational, and scientific developments remain integrated and aligned with future space exploration opportunities."Footnote 47

During the later portion of the period covered by the evaluation, the CSA did initiate a process that aims to centralize some of these technology development activities, in order to ensure greater coordination and efficiency. The precise scope of this centralization has yet to be settled and this process is expected to evolve well past the tabling of this report. As a result, the purpose of this sub-section is limited to sharing initial findings that relate to this topic.

As increasing efforts are being deployed to coordinate technology development activities within the CSA, it would appear helpful to more clearly distinguish between the types of technologies that are, in fact, funded through the STDP:

  • Generic technologies refer to those technologies that are applicable to multiple platforms, payloads, or ground infrastructures. More specifically, the CSA establishes a list of priority technologies where assistance is required in order to advance them along the Technology Readiness Level (TRL) scale, up to TRL 6.Footnote 48 Priority technologies are "those that have the potential for innovation and/or technological impact and have been established by the CSA as strategic technologies," based on the agency's current objectives.Footnote 49
  • Mission-enabling technologies are also based on priority technologies established by the CSA, and the goal of funded contracts is also to increase the TRL up to TRL 6. The key difference is that the mission-enabling technologies must already stand at a TRL 3 at the time of the contract, and priority technologies identified as part of a mission-enabling process are needed "for the approval and implementation of specific potential future missions of interest to Canada."Footnote 50
  • Finally, the technology development funded through contributions must relate to the development of industrial capabilities, including the development of novel concepts, products, or know-how for the purpose of increasing the commercial potential of Canadian space companies. The CSA does identify broad themes that are of strategic importance to Canada, and provides contribution funding accordingly.

While, technically speaking, there is a distinction between generic and mission-enabling technologies, the practical application of that distinction appears rather nebulous. Interviews conducted as part of this evaluation indicated that stakeholders, internal and external to the CSA, were at times uncertain about both the scope of each category and the relevance of the distinction between the two. In all cases, the technologies are selected by the CSA and must be related somehow to space activities that are considered or planned by the agency. Failing this, they would no longer be considered as a service provided to the CSA. Additionally, one could argue that a mission-enabling technology, while focussing on one potential mission, may well find other, more generic applications, and vice versa. Evaluation findings indicate that, in fact, more recent RFPs no longer apply this distinction.

Technologies developed through contributions are more readily distinguishable, as they must not result in the direct acquisition of any goods, services, or assets by the CSA.Footnote 51 Nevertheless, the CSA establishes the overall themes that must be addressed through these R&D activities, which are expected to contribute to enhancing the industrial capability of the Canadian space industry.

As centralization efforts are being pursued, the STDP is bound to operate in a more integrated framework, alongside other CSA technology development initiatives serving a variety of purposes. In so doing, its continued relevance will be largely established by its ability to maintain its current focus on supporting the planning of future missions and enhancing the capability of the Canadian space industry. This is seen as being particularly important among CSA sectors that are currently benefitting from STDP activities. Interviews indicate that the rationale for centralization appears to be well understood and supported, as long as any consolidation efforts do not dilute the type and range of assistance currently provided through the STDP.

Ultimately, this realignment is also expected to be reflected in the CSA's PAA. It is worth noting that, at the time of the evaluation, the description of the SSP only made reference to the development of generic technologies, thereby excluding any reference to mission-enabling or capacity-building activities undertaken through the STDP.Footnote 52

4.2.2 Efficiency of the IPMTT program structure

During the period covered by the evaluation, the organizational positioning of the IPMTT group shifted. Placed within corporate services in 2010, the group moved to the Technology Development Management directorate in 2012. Regardless of this change, its core mandate, which is to support the entire agency on any IP-related matters, remained the same.

4.2.2.1 Organizational positioning

Evaluation findings indicate that there is a sound rationale for positioning the IPMTT group close to technology development activities. During the course of interviews, key informants noted that other space agencies are adopting this approach in recognition of the direct link between IP rights and technology development.

Being closely linked to technology development does not mean, however, that IPMTT activities are to be limited to IP rights that are generated through STDP activities. As already noted, the CSA is engaged in a much wider range of technology development activities, all of which involve some dimension of IP rights. The IPMTT group is expected to support the entire range of activities in which the CSA is involved, which include all technology development activities.

In light of these considerations, one organizational challenge faced by the IPMTT group is the fact that it remains largely absent from the CSA's PAA. At the time of the evaluation, the IPMTT group was technically included within the SSP, but none of the IPMTT activities were in fact reflected in the description of the SSP provided in the PAA. And as noted, limiting the IPMTT to the SSP does not adequately reflect the range of activities it undertakes, even those activities related to space technology development.

Finally, it should also be noted that the description of the roles and responsibilities included in the CSA's Policy on the Management of Intellectual Property and Technology Transfer (2011) does not reflect how IP rights are currently being managed.Footnote 53 The CSA will need to address this issue in due time.

4.2.2.2 Adapting to new requirements

From an operational point of view, one of the most significant changes that occurred for the IPMTT group during the period covered by the evaluation was the shift in services resulting from the decision of the CSA to essentially end in-house R&D activities. While the IPMTT group has continued to manage IP assets created through CSA activities, it also became more directly involved in other processes, such as the review of applications submitted in response to AOs. The expertise of the IPMTT group has been sought in order to assess the commercial potential of R&D projects submitted.

Evaluation findings indicate that the current delivery structure of the IPMTT group has proven to be sufficiently flexible to adapt to this new policy environment and its associated requirements. Internal stakeholders consulted as part of this evaluation (which only include a portion of clients served by the IPMTT group) indicated that the role of the IPMTT group has been responsive; they did not identify any significant barriers in carrying out these new functions.

4.2.3 Performance measurement strategies

At the time of the evaluation, both the STDP and IPMTT groups had recently completed their respective performance measurement strategy. These documents provide updated descriptions of program activities and expected results. They also identify a range of performance indicators that could be used to support the ongoing management of program activities.

The key challenge that both groups are now facing relates to the implementation of these strategies. Because the range of activities undertaken by each group differs, it is worth addressing them separately.

4.2.3.1 The STDP performance measurement strategy

The STDP's performance measurement strategy does identify relevant indicators for the various activities, outputs, and outcomes that are included in its logic model. There are certain issues, however, that may require further attention.

  • There is a heavy reliance on UNITAS for the collection of performance data. While this evaluation did not include a thorough assessment of this tool, interviews indicate that there are limitations in terms of what this database can deliver.
  • The strategy covers all levels of program outcomes. While there are merits to collecting as much performance information and data as possible, resources should be largely focussed on collecting performance information relating to the immediate and intermediate outcomes. Formal evaluation processes are better suited to assess longer-term program outcomes.
  • Since the collection of data information relies on statements of outcomes, it may be appropriate for the STDP group to clarify some of the current outcomes included in its strategy. In particular, statements of outcomes should not relate to the undertaking of activities, but should speak directly to the anticipated change that occurs as a result of the activities and outputs. For instance, at the time of the evaluation, the logic model included the following immediate outcome: "Increase the understanding of technical challenges relating to space technologies and their solutions." This can either refer to the notion that the STDP will undertake distinct activities to increase that understanding, or refer to the fact that activities already undertaken by the STDP will contribute to achieving this increased understanding. In the latter case, the statement of outcome could read as follows: "Internal and external program stakeholders have a meaningful understanding of the challenges relating to space technologies and their associated solutions."

Beyond these considerations, the STDP group should address the more fundamental question of how it intends to report on its performance. The current strategy is silent on this key dimension. One traditional avenue is simply to include some of this information in the existing reporting that the CSA undertakes, such as its Departmental Performance Reports.

A more promising strategy to consider would be to report the achievements of the STDP through a distinct communication tool that could be widely distributed within and outside the agency. The STDP is, in fact, one of the high-profile programs of the CSA, which receives considerable attention. At this point, the public releases related to the STDP remain largely focussed on the announcements of funding that result from RFPs or AOs. The STDP group may wish to consider a more systematic distribution of information on actual program results, including illustrations of how this funding advances the mission of the CSA or contributes to the development of Canadian industrial capabilities in the area of space technologies. For instance, in addition to statistical information on the number of R&D projects funded or the range of technologies covered by this funding, the STDP group could provide snapshots of long-term results associated with specific projects. The ultimate goal is to ensure that any performance information that is collected is used in a meaningful way, within and outside the CSA. This information would also support any future evaluation activities.

4.2.3.2 The IPMTT performance measurement strategy

Through its recent performance measurement strategy, the IPMTT group has identified relevant indicators based on its logic model. Also, the group has been systematically collecting information on the range of activities it undertakes. This forms a solid foundation that should assist in addressing certain challenges in terms of the implementation of the performance measurement strategy and the communication of its performance results.

Like the STDP group, the IPMTT group would benefit from focussing on documenting its immediate and intermediate outcomes, with the view of using more formal evaluation processes to address longer term outcomes. Its logic model would also benefit from a review and clarification of its statement of outcomes, in order to avoid potential ambiguity between activities undertaken and expected outcomes.

At a more fundamental level, the IPMTT group may wish to review the articulation of its activities and expected outcomes to more closely reflect the fact that the group provides advice to internal clients of the CSA and that, ultimately, it is those internal clients that make decisions on the management of IP assets. As such, the extent to which the range of services offered by the IPMTT group meets certain standards of quality and usefulness would be outcomes to monitor. In addition, the level of awareness among CSA employees of IP matters, the level of CSA compliance with any legislative IP requirements, or the extent to which the advice provided is used to inform IP-related decisions would also be important aspects to cover. In the end, one would logically expect that sound IP management will support the achievement of the fundamental mission of the CSA, but the performance of the IPMTT group should focus on its capacity to effectively operate as an advisory entity.

Just as importantly, the IPMTT group will need to clarify how it intends to communicate its performance information. To date, it has issued internal reports that include some of its performance information. In accordance with its current performance measurement strategy, the IPMTT group is also planning to use case studies to illustrate how the CSA is reducing risks associated with IP matters and maintaining the continuum of the innovation chain.Footnote 54 Additional information of this nature could prove helpful in communicating the IPMTT group outcomes. Ultimately, adding the activities of the IPMTT group to the formal reporting structures of the CSA, such as its PAA, would allow its performance information to be communicated more broadly, through mechanisms that could include the CSA's Departmental Performance Reports.

4.2.3.3 Distinct or joined performance measurement strategies?

While the STDP and the IPMTT groups undertake different activities, they do fall under the same SSP and, as such, are expected to pursue similar goals. In more typical circumstances, only one performance measurement strategy would be developed for the SSP as a whole. As this evaluation illustrates, however, combining the two programs under the same strategy would be challenging, particularly as the activities of the IPMTT group extend beyond the STDP to include some activities that are completely distinct from the STDP. This again illustrates the need to clarify the positioning of the IPMTT group within the PAA of the CSA.

4.3 Performance

Within the parameters set by the methodology used for this evaluation, this section addresses the results achieved by the SSP during the period covered by the evaluation. The information is structured along the outputs and outcomes associated with the SSP.

Overall, evaluation findings indicate that the SSP has contributed to the strategic goals of the CSA, through the development of new technologies required by internal clients of the CSA, and by enhancing the capability of the space sector. The following sub-sections provide further information relating to the performance of each of the two components of the SSP.

4.3.1 The STDP performance

The performance of the STDP is explored through an analysis of the program's ability to deliver its expected outputs, and the results it has achieved. It also includes a discussion on the extent to which the program has been delivered efficiently.

4.3.1.1 Program outputs and outcomes

During the period covered by the evaluation, the STDP successfully launched six RFP processes for research contracts (STDP-03 to STDP-08), and two AO processes for contributions (including the pilot phase in 2012). In addition, the STDP group managed a number of directed contracts,Footnote 55 as well as ongoing activities that were launched prior to April 2010 but that were still active at the time of the evaluation. Finally, the STDP group managed research contracts in support of the Polar Communication and Weather (PCW) mission. A total of 125 contracts and agreements were active during the period covered by the evaluation. Table 4 provides further details on the number and value of these contracts, and the investments made during the evaluation period.

Table 4: STDP investments per categories, 2010-2011 to 2014-2015 ($)

ActivitiesFootnote 56 # of projects Total value Spending during the evaluation period
2006 STDP RFP (generic and mission-enabling technologies) 3 1,303,337 462,926
STDP-01 RFP (mission-enabling technologies) 15 9,221,758 4,923,733
STDP-02 RFP (generic technologies) 10 2,761,397 1,659,927
STDP-03 RFP (generic technologies) 1 445,863 424,632
STDP-04 RFP (mission-enabling technologies) 7 4,378,392 4,051,321
STDP-05 RFP (mission-enabling technologies) 11 9,002,454 5,818,747
STDP-06 RFP (generic technologies) 12 4,717,282 2,910,884
STDP-07 RFP (CASS-CATS) 1 1,351,917 737,716
STDP-08 RFP (TICFIRE) 1 747,338 649,990
STDP AO-01 (industrial capacity-building contributions) 8 679,674 679,674
STDP AO-02 (industrial capacity-building contributions) 38 13,065,238 1,491,805
Directed contracts 15 16,838,902 7,618,955
Support to the PCW mission 3 10,095,326 9,620,606
Total 125 74,608,878 41,050,916

Source: CSA administrative and financial data, February 2016.

It is estimated that 45 different providers of services have been involved in these projects, including 39 private entities and six universities.

These numbers speak to the success of the STDP in delivering its expected outputs. R&D projects have been undertaken to address the priority technologies identified by CSA internal clients (both generic and mission-enabling); close to $14 million have been invested through 46 projects to enhance the industrial capability of the Canadian space sector. As already noted in this report, both internal clients and external partners have been satisfied with the overall processes used to launch and manage these contracts and contribution agreements.

In terms of the expected outcomes of the STDP, evaluation findings confirm that the knowledge acquired through these activities has, in fact, increased the CSA's understanding of the range and nature of the technical challenges that potential missions could involve, and solutions that exist to address these challenges. For instance, administrative data confirm that, in practically all projects undertaken through the STDP, technologies have advanced along the TRL scale. Also, survey results and interviews confirm that internal clients have been in a position to identify and remove some or all uncertainties related to their respective technologies.

More specifically, CSA internal clients have identified a range of impacts associated with STDP research contracts. Surveys conducted since 2009 indicate that the technology development funded through the STDP has, in almost all cases, reduced major technological risks for potential missions. These R&D activities have also led to a range of performance improvement such as signal gain, data resolution, or pointing accuracy. The range of benefits further expands to cover improved reliability or longevity, improved technology compatibility, and a reduction in development or implementation costs.

Figure 4 provides further information from the survey of CSA internal clients and illustrates the range of impact that STDP research contracts have had.

It is also worth mentioning that, at the time the 2009-2012 STDP client survey was conducted, one of the 12 generic technologies funded had already flown (as technology demonstration) and another was adopted by an upcoming operational mission. Of the 22 mission-enabling technologies developed, nine were proposed for future named missions or for future technology demonstrations, five technologies were successfully adopted by actual missions, and one had already been launched.

Figure 4: STDP impacts per categories of potential challenges

Risk Performance Reliability Compatibility Costs Constraints Regulatory Agreements
Significantly beneficial 40% 40% 28% 27% 23% 30% 28% 8%
Somewhat beneficial 47% 45% 48% 45% 47% 32% 17% 5%

Source: STDP client surveys 2009-2012 and 2013-2015 (n=60)Footnote 57

Criteria Description
Risk Reduce major technological risks for a potential host mission(s)
Performance Improve performance (i.e., signal gain, data resolution, pointing accuracy)
Reliability Improve reliability or longevity of the technology
Compatibility Improve technology compatibility or system interoperability
Cost Reduce development or implementation costs
Constraints Reduce mass, power, or volume
Regulatory Gain independence from regulatory constraints (such as International Traffic in Arms Regulations - ITAR)
Agreements Permit adherence to international agreements or protocols (i.e., United Nations)

As for contribution agreements, interviews with industry representatives indicate that the experience to date (which was limited at the time of the evaluation) points to promising results. There is a perception that the range of themes included in AOs does reflect the needs of the industry, and the R&D projects funded do place the Canadian space industry on a stronger footing. Since the CSA's Class G&C Program requires that funding recipients submit yearly reports, additional information will become available to further explore the impacts of these contributions.Footnote 58

It must be acknowledged, however, that the extent to which these various technologies will achieve their full potential, will be placed on actual missions, or will be commercialized (when applicable) is an outcome over which the program has limited control. In this context and, as already noted, it might be beneficial for the program to produce specific snapshots (through case studies, for instance) in order to better document the complete story behind some of the technologies it supports, in order to highlight the STDP's contribution to the CSA's priorities and missions.

4.3.1.2 Program efficiency

Evaluation findings indicate that the STDP group has delivered its programming efficiently. While the level of activities within the program has increased during the period covered by the evaluation (which included the launch of a new component specific to contribution agreements), the overall level of FTEs has remained stable. Also, interviews conducted as part of the evaluation have not pointed to any significant issues related to the efficiency of the program.

One aspect that did raise challenges during the evaluation period relates to the ability of the program to amend its contracts when circumstances warrant such adjustments. As it currently stands, the STDP does allow for amendments, and specific financial resources are set aside to accommodate these. Proceeding with amendments is expected, considering the R&D projects funded are exploratory in nature. However, in at least one circumstance, the program has had to issue a full RFP to accommodate changes to an existing contract. This was done through the STDP-08 for TICFIRE. In practical terms, this RFP was issued to accommodate an opportunity to test the technology developed through this contract. Evaluation findings indicate that significant resources were invested to complete this step, which diverted resources from the actual project. An amendment to the contract would have been more efficient, but proved impossible within the current parameters set by the program and by PSPC.

4.3.2 The IPMTT performance

As already noted, this evaluation has focussed on the activities that the IPMTT group has undertaken in support of space technology development, and specifically those undertaken through the STDP. As a result, not all results achieved by the IPMTT group are included in this analysis.

4.3.2.1 Program outputs and outcomes

During the period covered by the evaluation, the IPMTT group has successfully delivered a range of services in support of IP management. For instance, in 2013-14, the range of services offered by the IPMTT group included (but was not limited to) the following components:

  • the management of the CSA's technology portfolio, which included 94 active technologies and 26 active patents;
  • follow-up activities related to 26 licences for commercialization and R&D, and 29 active trademarks;
  • activities undertaken to secure five new patents and eight new licences;
  • posting 36 technologies on web platforms (such as Sparkup, Flintbox, and the CSA's website).

The IPMTT group is asked on an ongoing basis to respond to a number of ad hoc requests. The number of these requests has constantly increased during the period covered by the evaluation. In 2010, the IPMTT group responded to approximately 115 of these requests; this number increased to approximately 160 in 2014-15.

The IPMTT group also undertakes a number of training activities to enhance the overall awareness and understanding of IP-related issues among the CSA staff. Some of these activities involve other partners of the CSA, such as the National Research Council and Justice Canada. Over the period covered by the evaluation, the IPMTT group provided 19 training sessions, involving 277 participants. Satisfaction levels among participants, as reported through feedback questionnaires, were very high. As Table 5 indicates, these sessions have covered a range of IP-related issues.

Table 5: IPMTT training activities (2010-2011 to 2014-2015)

Themes # of sessions # of participants
Copyright related issues 1 34
Overview of IP 3 47
IP related to the RCM initiative 1 27
IP in agreements 2 18
IP in contracts 3 30
IP and the media 2 17
IP and online content 2 43
Patent related issues 3 26
IP and software 2 35
Total 19 277

Source: IPMTT performance data, October 2015.

To support the commercialization of CSA's technology portfolio, the IPMTT group contracts a number of studies whose purpose is to assess the market potential of specific technologies. Since 2013, 15 studies have been completed, and one of these has led to the signature of a licence with a Canadian software company.

In addition to updating CSA's policy on IP, the IPMTT group has initiated a number of activities relating to the management of IP in contracts signed by the CSA, as well as activities that facilitate technology transfers in order to maximize their commercial use.

All interviews conducted as part of this evaluation point to a high degree of satisfaction with the services provided by the IPMTT group. In particular, key informants noted that the group has successfully managed a number of challenges related to the management of background and foreground IP, and has established an efficient process to facilitate technology transfers for space and non-space related R&D projects. In addition, the IPMTT group has successfully added new dimensions triggered by the implementation of the contribution component of the STDP, such as the assessment of bids and applications received to better understand their commercial potential.

4.3.2.2 Service efficiency

The level of resources attributed to the IPMTT group has remained stable throughout the period covered by the evaluation. Evaluation findings indicate that the IPMTT group has succeeded in carrying out its mandate and adjusting to new demands. Evaluation findings have not pointed to any specific concerns related to the efficiency of the current service delivery structure.

5. Conclusions and Recommendations

5.1 Relevance

The development of new technologies stands at the core of the CSA's ability to actively engage in space activities. Throughout the period covered by the evaluation, a number of reports and studies urged the federal government to enhance the CSA's ability to carry out space technology development in a more systematic and predictable manner. It was also recognized that such a goal can only be achieved through the direct involvement of a strong and sustainable Canadian space industry.

While the CSA has a number of programs that support space technology development, the STDP continues to be the main tool through which space technologies related to future space mission opportunities are developed. Without the STDP, it is unlikely that the CSA would be in a position to maintain its current involvement in space activities. It is also through the STDP that the Canadian space sector can access the type of support it requires to develop technologies that enhance its industrial capability. As much as this statement speaks to the relevance of the STDP, it also illustrates the fact that the federal government has yet to implement a strategy that would engage a broader range of federal partners in supporting space technology development. Programming offered by other federal departments or agencies either remains limited in scope, or does not easily accommodate the nature of space technologies.

The predominant role that the private sector plays in conducting space technology development reflects a historical trend within the CSA; it also reflects practices in other space agencies around the world. Over the years, only limited R&D activities have been undertaken within the CSA. During the period covered by the evaluation, the CSA essentially ended these activities as a result of a strategic review process. Early signs indicate that this approach may prove problematic over time, as in-house subject matter expertise is not being renewed. It is essential that the CSA maintains its ability to monitor and support the work done through procurement processes.

Recommendation #1: The CSA should explore avenues to sustain the level of in-house expertise required to ensure the proper management of space technology development.

The SSP also provides the framework needed to manage the range of IP requirements associated with space technologies. It is recognized that only through an effective management of IP can the CSA ensure that the technologies it supports benefit the CSA and the broader Canadian society.

5.2 Program design

The SSP benefits from a sound program delivery structure covering both STDP and IPMTT activities. For the two components, the CSA has established efficient processes and structures that have proven capable of undertaking the range of activities covered by their respective mandates.

As it relates more specifically to the STDP, it must be acknowledged that the absence of a long-term plan establishing space priorities is limiting the ability of the program and its internal clients to plan technology requirements. More specifically, the evaluation confirms that the STDP is capable of undertaking the research required by internal clients, and the TPWG has proven useful in that regard. But the broader question of whether the STDP will deliver the required technologies at the appropriate time to allow the CSA to implement its broader agenda cannot be fully addressed in the absence of such a plan. This issue, while certainly relevant to the STDP, does fall beyond the scope of this evaluation.

Since processes that lead to the issuance of RFPs (for research contracts) and AOs (for contributions) involve multiple steps and internal stakeholders, it can be challenging for the STDP group to plan with precision the release of these RFPs and AOs and, thus, inform external partners who may be in a position to respond and submit bids or proposals. While recognizing these factors, it must be noted that any gains that could be achieved in making these releases more predictable and easier to manage for external stakeholders would benefit the program.

Recommendation #2: The CSA should explore avenues to enhance the planning and communication activities leading up to the releases of Requests for Proposals and Announcement of Opportunities, in order to ensure predictable funding timelines and enhance the ability of the Canadian space community to respond to these calls.

The efforts initiated during the period covered by the evaluation to consolidate a portion of space technology development carried out by the CSA will provide an opportunity to clearly establish how the STDP is positioned in relation to other space development programming. To this end, the CSA should ensure that its PAA (and the associated reporting) more clearly reflects the range of activities undertaken by both the STDP and the IPMTT groups, as the latter is absent from the current PAA.

Recommendations #3: The CSA should review its current Program Alignment Architecture (PAA) in order to appropriately reflect the range of activities undertaken by the STDP group to support generic and mission-enabling technologies, and to enhance the industrial capability of the space sector. The CSA should also ensure that the next PAA revision appropriately reflects the range of activities undertaken by the IPMTT group.

The STDP and the IPMTT groups have developed performance measurement strategies that provide helpful information on the activities they undertake and the results they are aiming to achieve. This evaluation has identified a number of avenues that could be explored to enhance these strategies. In particular, the CSA should clarify how it intends to report the performance information it is expected to gather.

Recommendation #4: The CSA should review the current performance measurement strategies for the STDP and the IPMTT groups, based on the opportunities for improvement identified in the evaluation report, including strategies to more effectively communicate actual SSP impacts and how they contribute to the CSA mission.

5.3 Performance

The two components of the SSP have successfully implemented the range of activities required by their respective mandate. The STDP group has launched several new research contracts, in addition to managing ongoing R&D projects and implementing the contribution components of its mandate. The IPMTT group has provided a range of services that have ensured a sound management of IP within the agency, in addition to providing additional support for the implementation of the contribution component of the STDP.

As a result of these achievements, the CSA has enhanced its understanding of the range and nature of the challenges associated with potential missions, and the promising avenues that exist to address these challenges. The CSA is also more readily engaged in enhancing the capability of the Canadian space industry, making it better positioned to engage in projects led by the CSA or by other agencies or entities, and to undertake commercial endeavours as applicable.

The evaluation also indicates that the SSP is being delivered efficiently. The STDP group has successfully carried out a broader mandate, particularly as it relates to the implementation of the contribution components, while keeping the level of operational resources fairly consistent. The program appears to be sufficiently flexible to accommodate unforeseen developments, although it may be worth exploring whether further flexibility could be accommodated to ensure that RFPs or AOs remain focussed on their original purpose, namely to explore a range of priority technologies based on planning activities. The IPMTT group has also managed an increase in its overall level of activities, including both an increase in ongoing services and initiatives specific to the identified missions of the CSA or new IP requirements.

Management Response and Action Plan

  Responsibility
Organization / Function
Management response Details of action plan Schedule
Recommendation # 1
The CSA should explore avenues to sustain the level of in-house expertise required to ensure the proper management of space technology development. Director General, Space Science & Technology supported by Director General, Space Utilization, Director General, Space Exploration and Executive Director, Corporate Services and Human Resources CSA will explore avenues to sustain the level of in-house expertise. The expertise will be in-line with Canada's Space Policy Framework to support technologies in SAR, Robotics, Satellite Communication and Optics in addition to emerging technologies.

Sustain and progressively improve where appropriate and feasible the involvement of CSA highly qualified personnel supporting R&D by:

  1. clarifying the role of the experts and performing a review of technical and scientific competencies needed to support R&D in the long term and across programs and missions in CSA;
  2. identifying and proposing specific initiatives to maintain a high level of technical and scientific skills such as collaborations and exchanges of personnel with other R&D organizations in Canada and abroad or different involvement in CSA activities.
  1. March 2017
  2. March 2017
Recommendation # 2
The CSA should explore avenues to enhance the planning and communication activities leading up to the releases of Requests for Proposals and Announcement of Opportunities in order to ensure predictable funding timelines and enhance the ability of the Canadian space community to respond to these calls. Director General, Space Science & Technology We agree that an effort should be made to make RFP's and AO's more predictable and that CSA should inform the space community prior to the release of RFPs and AOs. As part of the STDP review planned for 2016-2017 in collaboration with stakeholders, we will revise RFP and AO processes to study the possibility of making RFP's and AO's more predictable. March 2017
Recommendation # 3
The CSA should review its current Program Alignment Architecture (PAA) in order to appropriately reflect the range of activities undertaken by the STDP group to support generic and mission-enabling technologies, and to enhance the industrial capability of the space sector. The CSA should also ensure that the next PAA revision appropriately reflects the range of activities undertaken by the IPMTT group. Executive Director, Programs and Integrated Planning supported by Director General, Space Science & Technology While we support the recommendation, TBS is reviewing the approach on reporting results. This review will have an impact on the revised results framework. We will change the related section in the results framework in accordance with TBS new policy requirements. November 2017
Recommendation # 4
The CSA should review the current performance measurement strategies for the STDP and the IPMTT groups, based on the opportunities for improvement identified in the evaluation report, including strategies to more effectively communicate actual SSP impacts and how they contribute to the CSA mission. Director General, Space Science & Technology While we support the recommendation, TBS is currently reviewing its policy on performance measurement therefore we will wait for the new guidelines before reviewing STDP's and IPMTT's Performance Measurement Strategies. Note that the two programs already started the implementation of better communications mechanisms in 2015. For example, IPMTT publish a yearly report on their activities. STDP started last year to produce high level summary for contracts given under RFP 10.
  1. The CSA will review the performance measurement strategies for the STDP and IPMTT groups based on the new TBS guidelines and including an approach to more effectively communicate their impacts to stakeholders. Therefore although efforts will be made in 2016-2017, final version is not expected before 2017-2018.
  2. Since a strategic review of the STDP is planned in 2016-2017, in collaboration with stakeholders, it provides another opportunity to improve the program and communicate its results.
  1. March 2018
  2. March 2017

Appendices

Appendix A: Logic model of the STDP

This appendix provides a description of the various components of the STDP's logic model, illustrated in Figure 5. This narrative is based on the STDP's performance measurement strategy.Footnote 59

Activities

Allocate available budget according to provided needs and opportunities (A1)

STDP receives its O&M and contributions budget (after 2012) at the beginning of the fiscal year. Before 2014, the STDP budget was determined by the Executive Committee and fluctuated over the years.

The STDP is provided with prioritized lists of technologies deemed to require development. For those related to CSA needs, the suppliers of the lists are STDP's internal clients, namely SE, SU and SS&T. SE and SU General Directorates base their respective lists on their mission roadmaps and their associated technical requirements. The funds are then attributed to the government needs via the contracts procurement mechanism or to the industrial needs via the contributions mechanism.

The budget allocation (distribution), for the government needs, has been done several ways during the last five years. RFPs and AOs are written in response to the priorities established and in consideration for the budget available. For certain RFPs, the budget was divided between mission-enabling technologies and generic technologies, with mission-enabling receiving the larger share of the available funds. More recently, both technology types have been combined into a single prioritized list rather than two separate ones.

Regardless of the number of lists used to capture the ranked/prioritized technologies, the process of deciding how many technologies to fund (and thus include in the host RFP) remains the same and includes as many technologies as the available budget will allow starting with the top-ranked technology and working down the list (or lists).

The same formula is applied for contributions in support of industrial capacity-building.

Figure 5: Logic Model — Space Technology Development Program (STDP)
Logic Model - STDP
Text description of image for Figure 5

Inputs

  • Funding as per SAP/IPS, FTE, Lists of technologies (generic technologies, mission-enabling and industry capabilities) to be developed from SE, SU and industry.

Activities

  • Allocate available budget according to provided needs and opportunities (A1)
  • Identify and prioritize capacity-building opportunities (A2)
  • Plan procurement (A3)
  • Implement and manage R&D initiatives (A4)

Outputs

  • Issued contracts for the development of generic technologies (Op1)
  • Issued contracts for mission-enabling technologies (Op2)
  • Issued contributions for the development of industrial capabilities (Op3)

Immediate Outcome

  • Increase the understanding of technical challenges relating to space technologies and their solutions (Oc1)

Intermediate Outcomes

  • Reduce technical uncertainties and unknowns linked to space missions and/or activities (Oc2)
  • Increase the number of innovative generic technologies that have the potential of contributing to space-related endeavours (Oc3)
  • Increase/maintain technological capacity of the Canadian space sector (know-how, processes and/or products) (Oc4)

Ultimate Outcome

  • Enhance the Canadian space sector's ability to respond to Canada's current and future space objectives as well as its ability to be competitive in the global market (Oc5)
Identify and prioritize capacity-building opportunities (A2)

A newer mechanism to fulfill the TDM Directorate's capacity-building mandate is the CSA Class G&C Program. This contribution mechanism is a new one for SSP Enabling Technology development-STDP element, with the pilot AO released in 2012 and the second one (AO2) released in June 2014.

Since 2012, the TDM funds basic R&D in key technology areas through non-repayable contribution agreements. The responsibility for managing these contributions and their associated technology development is assumed by the STDP group since STDP is CSA's centre of expertise for managing industry R&D initiatives.

The technology themes funded through non-repayable contributions are identified by CSA's industrial partners. The second AO (AO2), posted in June 2014, emanated from consultation with the Aerospace Industries Association of Canada (AIAC) which spoke on behalf of the Canadian space industry at large. The starting point used by AIAC consisted of the priorities mentioned in the space policy framework, to which additional technology themes were added after consultation with the industrial partners that it represents.

Plan procurement (A3)

This activity pertains to establishing a procurement strategy for the development of priority technologies via contracts as well as the enhancement of industrial capabilities via contributions. The contract route involves working closely with PSPC (since June 2014, STDP pays for a PSPC dedicated resource) and the technology clients to develop a Request for Proposal including a detailed Statement of Work with technical and management requirements. For the contribution route, it involves working closely with the CSA Class G&C Program's Centre of Expertise to develop AO for contribution initiatives.

Implement and manage R&D initiatives (A4)

This activity involves evaluating bids received in response to an RFP or an AO and awarding contracts or contributions to winning bidders and subsequently managing the R&D contracts or contributions. This involves STDP project authorities working closely with CSA technical experts and with contributions recipients or selected contractors, to ensure that the terms and conditions of the RFP or AO are met. At the close of the contract or contribution, the results are captured and disseminated to the CSA technology clients. Performance indicators for the STDP program are collected and stored in the information management tool (Livelink) for analyses during the course of the initiative.

Outputs

Issued contracts for the development of generic technologies (Op1) and of mission-enabling technologies (Op2)

The result of the RFP process is the awarding of contracts to the Canadian space sector (primarily to the Canadian industry but also to the academic sector to a smaller extent), the number of which depending on the O&M budget available.

It must be noted that the number of priority technologies listed in an RFP does not necessarily coincide with the number of contracts awarded. This is due to two main reasons. The first is that certain priority technologies do not receive any bids from the industry, while the second is that the bids received for certain PTs do not always meet the minimum passing mark during the evaluation process and are thus discarded. As a result of these two instances, it has been common practice to either award fewer contracts than anticipated or to award more than one contract for those priority technologies for which competing solutions are of interest to the STDP clients (SE & SU).

The STDP strives to maximize the utilization of its O&M budget by funding as many R&D initiatives as it can.

Issued contributions for the development of industrial capabilities (Op3)

The result of the AO processes is non-refundable (based on the exception "basic R&D") contribution-funded initiatives; the number of contribution agreements depending on the available contribution budget.

As is the case with R&D contracts, the number of basic R&D projects ending up being funded through contributions depends on budget availability. Although an AO may suggest general themes (such as Robotics and Satellite Communications), there is no guarantee that industrial partners will submit proposals for each theme. Moreover, certain themes may receive multiple proposals on separate technologies related to that theme. After all received proposals are duly evaluated; the end result typically amounts to certain themes without any funded projects, while others seeing multiple projects. Consequently, as per the contract situation described earlier, the number of projects being funded ultimately depends on the response received from bidders and on budget availability.

The STDP always strives to maximize the utilization of its contribution budget by funding as many initiatives as it can.

One last factor common to both RFP-derived contracts and AO-tied contribution agreements is the unknown financial aspect of the bids. Although cost maximums are specified for each Priority Technology and cost categories are specified for the industrial themes, the STDP does not know beforehand what the costs will be proposed by the applicants in their bids nor can it predict what the actual costs will be. These costs are only made known after PSPC has concluded its negotiations (in the case of contracts) and after the CSA's G&C Centre of Expertise has completed its own review of the cost estimated by the industrial partners (in the case of contribution agreements).

Outcomes

Immediate Outcome
Increase the understanding of technical challenges relating to space technologies and their solutions (Oc1)

The chief reasons for investing in Research & Development at CSA, through generic and mission-enabling technologies are to obtain as clear an assessment as possible of the technical challenges being presented by a certain mission and to determine the feasibility of each available option. This understanding is gained by both the CSA's mission champions (they include the internal clients either leading a CSA mission or advocating to have a Canadian technology aboard an international mission. They also include external clients who are leading a mission or who have a vested interest in knowing which unknowns and uncertainties that have been retired and if not why), and the CSA's industrial and academic partners to whom the R&D work is contracted out.

The gains obtained from the perspective of the industrial partners vary from having concluded that the proposed solution is either not possible or practical to having confirmed its feasibility. In so doing, the industry comes out with new knowledge about the technical subject researched and can even find itself with a product with eventual commercial potential in the process.

Intermediate Outcomes
Reduce technical uncertainties and unknowns linked to space missions and/or activities (Oc2)

Finding answers to technical unknowns and clarifying uncertainties with respect to technical feasibilities and practicalities are the first steps toward the CSA being in a position to decide if a mission is worth pursuing and if further development is warranted. In overcoming technical challenges related to space technologies, space missions that use the advanced technologies stand to gain in improved performance and in risk reduction. The recent client survey confirms that the technology developments do indeed provide such answers and do clarify uncertainties.

For the mission champions the results of the technology development provide them with answers to technical questions that can serve to arrive at any of the following conclusions and potential follow-up actions:

  • The technical challenges are too complex and resolving them is either not affordable or would be too lengthy an investment. Consequently, it is concluded that no further work will be pursued;
  • The obtained results demonstrate that the proposed technical solution is a viable one and can ultimately lead to a successful mission, mission system or mission sub-system although further development is required to bring the solution to the adequate level of maturity; and
  • The obtained results satisfactorily answer the technical unknowns and/or remove the technical uncertainties. Consequently, no further development is required at this time to allow the mission lead to complete the mission business case or to enter into final negotiation with an international mission lead for a Canadian participation in that mission.

The SSP posits that increasing the understanding of technical challenges pertaining to space technologies (Oc1) will lead to reducing technical uncertainties and unknowns. Ways of measuring this reduced uncertainty is demonstrated via the finalization of estimates (reducing financial uncertainties) and business cases (reducing commercial uncertainties) or establishment of partnerships agreements (reducing unknowns).

Increase the number of innovative generic technologies that have the potential of contributing to space-related endeavours (Oc3)

Some generic technologies in need of development are not tied directly to a specific future mission. If successful, those generic technologies can benefit a wide array of missions that can be unrelated to the needs of the government or can benefit the industry through advancing the competitiveness of the space sector. Examples of generic technologies are, but are not limited to, space subsystems, solar panels, data storage, navigation systems and the like.

Once developed to the point of providing sufficient confidence in the technologies, further development can be pursued by the Canadian industry, who may wish to bring it to commercialization, or by anyone else who wishes to obtain the IP rights to the technology, or by the CSA, through additional contracts to the industry.

Increase/maintain technological capacity of the Canadian space sector (know-how, processes and/or products) (Oc4)

The STDP's investments in support of basic R&D play an important role toward increasing the Canadian space sector's ability to respond to Canada's and international needs in space. This outcome is accomplished through an overall increase in knowledge regarding space-related technologies, applications and capabilities. This know-how can extend to the creation of new design and manufacturing processes as well as the creation of new products. This latter is particularly true for our industrial partners who not only benefit directly on this front from CSA's contribution financing but who can also derive commercial spin-off products from R&D funded through the contract mechanism.

Noteworthy of mention is the reality that the political and economic cycles (e.g., level of government and commercial investment in innovation) which affect Canadian industry may hinder the potential to build capacity.

Ultimate Outcome
Enhance the Canadian space sector's ability to respond to Canada's current and future space objectives as well as its ability to be competitive in the global market (Oc5)

All of the above-mentioned activities, outputs and outcomes aim to culminate to the ultimate outcome of « Enhancing the Canadian space sector's ability to respond to Canada's current and future space objectives as well as its ability to be competitive in the global market". The theoretical rationale behind this claim is based on the following logic:

  • In order to best position the Canadian space sector in its desired ultimate ability to respond to future Canadian space needs, the appropriate number, size and type of contracts and contribution agreements need to be planned for, budgeted and managed.
  • The understanding of these challenges and the eventual determination of innovative solutions are achieved via R&D activities funded either via contracts or contributions agreements.
  • Through the reduction of risks pertaining to space missions and the increase in the number of generic technologies as well as through the increase of technology capacity, it is inferred that the Canadian space sector (mainly the industry but also academia) is in a position of being equipped with additional knowledge and improved processes by virtue of having devoted time and expertise answering questions and responding to challenges associated to unique technological difficulties encountered when devices need to function in space. With these R&D activities, culminating in innovative solutions and potential commercial products, the space sector gains in its ability to being technically ready to respond to the Canadian government space objectives while increasing the industry's competitive edge on the global space market.
  • Further to Canadian industrial and academic partners gaining an understanding of the technical challenges at hand and, on identifying innovative potential solutions to these challenges, it is assumed that the space sector gains ability to respond to Canada's current and future space objectives as well as its ability to be competitive in the global market.
  • All of the above outputs and outcomes start with the identification of anticipated technological needs. For governmental needs, the STDP relies on requests emanating from clients' roadmaps for future missions and their associated technology development. For commercial needs, the STDP solicits inputs from the industry.

Appendix B: Logic model of the IPMTT group

This appendix provides a detailed description of the various components of the IPMTT's logic model illustrated in Figure 6. This narrative is taken from IPMTT's PM StrategyFootnote 60.

Activities

IPMTT consultation services for CSA staff (A1)

IPMTT supports all CSA sectors regarding IP questions through a single window approach consultation service. These demands and questions range from copyrights law application questions to recommendation regarding IP clauses in procurement contracts and general counseling concerning IP strategy for all CSA activities.

Awareness for IP matters for CSA staff (A2)

IPMTT elaborates and delivers training sessions to increase the CSA staff awareness of IP issues that can arise in the course of their duties. IPMTT developed an online training session with the help of the Canada School of Public Service (CSPS) that can be taken by all CSA employees and affiliates. That training session is presently hosted on the CSPS servers. Every new employee receives an invitation to complete this training session. This online training session is a primer on IP question that aims at familiarizing CSA staff with IP issues and provides them with a basic IP knowledge.

Moreover, traditional, in-class, training sessions are also created and delivered (an average of 2 per year). These training sessions are opened for all CSA staff and target specific IP matters that are of general interest (IP in contracts, IP in communications tools, patents, etc.).

Finally, CSA sector or CSA group specific training sessions are also devised and delivered upon request of the concerned group or sector. For example, IPMTT delivered to the RCM group a specific training session regarding RCM's contracts and IP regime.

In complement of the training sessions, IPMTT conducts regular internal communications activities. Monthly information capsules are broadcasted to all CSA staff via the internal communications screens and the intranet. Also, the IPMTT highlights the annual World Intellectual Property days by planning fun and informative activities on April 26th of each year.

Figure 6: Logic Model — Intellectual Property Management and Technology Transfer (IPMTT)
Intellectual Property Management and Technology Transfer
Text description of image for Figure 6

Inputs

  • Funding as per SAP/IPS, FTE, CSA portfolio of IP elements - SE, SU, SS&T and corporate issues on IP

Activities

  • IPMTT consultation services for CSA staff (A1)
  • Awareness for IP matters for CSA staff (A2)
  • Management of portfolio of technologies and licenses (A3)
  • Technology transfer activities (A4)

Outputs

  • Delivers advices and guidance regarding IP matters to CSA staff (Op1)
  • Performs identification, promotion, brokering activities for Crown-owned technologies (Op2)
  • Delivers training sessions on IP-related matters for CSA staff (Op3)

Immediate Outcomes

  • Guarantees access rights for government and non-government mandate (Oc1)
  • Establishes protection strategies to ensure government and non-government mandate (Oc2)

Intermediate Outcome

  • Manages Crown intangible assets to reduce risks associated with IP matters and maintain continuum of innovation chain (Oc3)

Ultimate Outcome

  • Facilitates the circulation of space technologies and products for space and terrestrial applications for the benefit of Canadians (Oc4)
Management of portfolio of technologies and licenses (A3)

IPMTT manages all the IP assets generated by CSA activities. These assets include copyright, trademark, official mark, industrial secrets and patents (design or utility). This IPMTT activity is framed by the Policy on the Management of Intellectual Property and Technology Transfer at the CSA. This policy is still active but needs updating since the IPMTT has moved from Corporate Services to SS&T, making some elements of the policy obsolete.

The management of those assets involves the collection of the relevant information and the archival of this information in the IPMTT dedicated IP database. Information related to inventions and their disclosures, as mentioned in the CSA IP policy, is covered under the Public Servant Inventors Act. This federal law applies to all federal public servants and mentions that all invention made by a public servant, whether during the course of its duty or not, must be declared to the relevant minister. For CSA employee, the industry minister has delegated this duty to the DG SS&T which, in turn, has mandated the IPMTT to manage the whole process. By extension, all internal IP management has been assigned to the IPMTT.

Once an IP asset is identified, it is evaluated to determine what would be the best scenario to manage it for the benefit of all Canadians. Disclosure, publication or protection strategies are elaborated. When it is concluded that technologies should be protected by patents, the IPMTT also manages the protection process. This process involves filing the patent applications in the relevant countries, answering to all patent authorities questions and paying the mandated application and maintenance fees.

This activity also covers the management of the licenses that were granted to third parties for the use of CSA technologies during Technology Transfer activities (see A4 below). The management of the licenses involves the monitoring of all parties obligation, CSA's and licensees', including, when applicable, the follow-up on the royalties that should be paid to the Crown and the gathering and filling of the licensees mandated reports.

Technology transfer activities (A4)

IPMTT has the mandate to identify and facilitate the transfer of Crown-owned technologies (either Crown-invented or third-party invented but transferred to the Crown through procurement contracts) to third parties to allow commercial exploitation of those technologies. The rationale behind this activity is that commercial exploitation is an effective way of deriving benefits for Canadians from Crown R&D activities. Indeed, commercial exploitation can create jobs, solidify existing Canadian companies, encourage the creation of new companies and, at the very least may lead to new products being available for Canadians.

To successfully transfer an IP asset for commercialization, the IPMTT must publicize the fact that the relevant IP is actually available for transfer. To do so, the IPMTT posts its available technologies on different web platforms and produces marketing literature (commercial one-pagers) that can be sent to interested parties. Since mid-2015, as a service to the space sector, the IPMTT also give the possibility to publicize third-party owned technologies funded by CSA through its procurement contracts.

Commercial exploitation also necessitates a good knowledge of the relevant markets. Therefore this activity includes market information gathering. This knowledge, in turn, allows the IPMTT to elaborate a relevant protection strategy or, when needed, a disclosure strategy with the help of the inventors and their chain of command. To obtain this market intelligence, the IPMTT, with the help of consultant firms, conducts market assessment studies following a three phases methodology developed by the IPMTT team. In broad terms the three phases are as follows:

  1. Market identification
  2. Market validation
  3. Preparation of brokering activities

After the market assessment studies, the IPMTT enters, when the conditions are favourable, in brokering with interested parties. The IPMTT negotiates the conditions of the license and is in charge of the internal approval process. Once a licensed is signed, the monitoring of the obligations of that type of agreement is included in A3 (see above).

Outputs
Delivers advices and guidance regarding IP matters to CSA staff (Op1)

The IPMTT Op1 consists of delivering different kinds of advices and guidance pertaining to IP. The nature of those requests varies from concerns about disclosing IP elements owned, or not, by the Crown to third parties to interpretation of contractual provisions regarding IP rights. This increase can be explained by the increased awareness of the CSA staff toward IP issues. Since 2010, the volume of requests for IPMTT consultations by CSA staff went from 115 to 160 advices provided in 2014-15.

The IPMTT summarizes the types of advices and guidance it provides in three broad categories related to the questions that prompted the advices or guidance: Major Crown projects related questions, R&D contracts related questions and all other IP-related questions. The amount of work necessary to deliver the required advices or guidance varies according to the nature of the questions. Major Crown project related questions often necessitate a large amount of work compared to the other types of questions.

Performs identification, promotion, brokering activities for Crown-owned technologies (Op2)

As mentioned in A3, IPMTT is responsible to manage IP owned by CSA (Crown). The volume of Crown-owned IP has been decreasing since the beginning of the evaluation period (2010-2015), but has not completely disappeared. The decrease is mainly seen after 2013, following the decision by the CSA management to cease all in-house research activities. This decision was in line with some recommendations from the Emerson Report to ensure that most of the R&D activities in the space sector be done in the industry or in academia. This change had impact on the volume of Crown-owned technologies that are being developed.

Some CSA employees are still inventing new concepts and technologies as a part of their work in 2015. Moreover, the legacy IP portfolio that was built throughout the years (patent life is usually 20 years) still needs to be managed appropriately.

Once an invention disclosure is completed, an IPMTT officer acknowledge its reception within two weeks and conducts its evaluation within 6 months. This first evaluation usually serves at identifying the commercial potential of the technology as well as establishing the preliminary protection strategy for the technology. The IPMTT officer will also disclose the commercially interesting inventions to the public by posting the technologies in question on the relevant web platforms when it is appropriate to do so. To post a technology the IPMTT officer first evaluate the commercial timing of the post and the protection status of the technology (unprotected proprietary information is not posted). The web platforms presently used by the IPMTT are: The CSA public website, Flintbox and Intellectual Property Exchange.

Delivers training sessions on IP-related matters for CSA staff (Op3)

IPMTT is responsible to give training on IP matters to all CSA employees. As discussed in Op 1, the IPMTT must answer to a growing number of requests. The IPMTT offers IP training sessions to all CSA employees to allow them to better understand simple IP issues and also identify the more complex IP issues to be deferred to the IPMTT staff. This allows some IP issues to be resolved without the need for an IPMTT consultation and ensure that all necessary IP issues are being managed properly at the CSA. The IPMTT developed an optional online training session on the basics of IP that is available to all of CSA employees. The training is online, free and offered to all new employees, involved or not with scientific and technological activities.

Formal training sessions pertaining to IP matters are offered on a regular basis to the CSA staff. Subjects vary in function of the requests for consultation received or of specific needs. For instance, a training specific to the IP regime of RCM was offered to the RCM staff in 2014-15.

Outcomes

Immediate Outcomes
Guarantees access rights for government and non-government mandate (Oc1)

IPMTT supports all the CSA Branches to ensure that Canada is granted the IP rights that it needs to support its mandate and priorities. For instance, IPMTT provides the expertise and tools to identify the IP that was developed during activity (e.g. a R&D contract) and the pre-existing IP for which rights are needed to exploit the newly developed technology. It also interacts with external collaborators when issues arise about IP ownership or rights as mentioned in Op1. IPMTT also trains all CSA staff to be more aware of IP matters and empowers them to make better IP-related decisions as mentioned in Op3.

Together, these three activities allow the CSA and its staff to incorporate IP questions in all of its processes ensuring that these processes will not be disturbed by IP matters. For example, including IP issues in early decision making process can help put in place effective procurement strategies that will not only allow the CSA to fulfill its own objective for the contract, but will also allow the industry to benefit from it down the line. Indeed, a good IP strategy will allow for a competitive environment where all industrial partners can thrive, not only the winners of a few basic contracts. Moreover, a sound IP strategy will also give the opportunity to commercialize the outcome of the contracts, thus making the contracts more interesting for the companies that bid on them by giving those companies another incentive on top of the total funding being paid by CSA.

Non-commercial entities also benefit from a sound IP strategy. For example, when IP is wisely managed, universities can ensure that their professors can publish the results of the work they do for or in partnership with CSA. Moreover, universities can also ensure they can teach the results of such work to their students; therefore multiplying the effect of such work and contributing to the creation of HQPs in Canada.

Sound IP management allows CSA to ensure that it will be able to fulfill its objective. IP management allows the CSA to share critical information with its international partners such as the ISS partners. It allows the CSA to contribute critical pieces of technology to important space missions. It also allows for the CSA to guarantee it will be able to fully use the infrastructure it paid for to fulfill its mission. For example, source code of critical software belonging to CSA contractor is put in escrow to ensure Crown access in the event the company is unable or unwilling to share this information with the Crown.

Establishes protection strategies to ensure government and non-government mandate (Oc2)

The decisions, expenses and actions related to specific IP matters are documented on a roadmap and the documents are in the CSA IP database as mentioned in Op2.

Protection strategies consist of building an IP portfolio that:

  • Is marketable to be attractive for the industry.
  • Is strong enough to ensure that CSA will be able to continue using its content without being vulnerable to third parties IP claim.
  • Includes timely public disclosure. IP that is disclosed too early is not patentable and IP that is never disclosed cannot be used by third parties. Disclosure can be made via patent publication or scientific publication.

The IPMTT ensures that CSA IP is disclosed at the optimal moment to maintain its value, both commercial and academic.

A marketable portfolio is desired to be able to transfer these government assets to the private sector for the benefit of all Canadians and thus ensuring that government funded technologies can be accessible to the population and be a driver of commercialization and socio-economic benefits. The private sector often perceives value when a technology is protected. Licensees of CSA technologies use the Crown protected IP to fulfill their commercial objective. The Crown also uses its IP portfolio to ensure it will be considered as a credible partner in international space missions.

A strong portfolio also prevents the Crown from being targeted for using technologies it developed, thus ensuring that the CSA will be able to maximize the use of its IP assets.

The receipt of a Contractor's Disclosure of IP that is owned by the Crown or of an invention disclosure by a CSA employee triggers a series of activities that will translate in a protection strategy of this IP.

Intermediate Outcome
Manages Crown intangible assets to reduce risks associated with IP matters and maintain continuum of innovation chain (Oc3)

Through the effective stewardship of the intangible asset that is IP, the IPMTT places the Canadian space sector (CSA, industry and academia) in a position of maximizing the use of the IP it developed and prevent the loss of resources linked with IP issues.

Together, Oc1 and Oc2 allow both the CSA and its partners, both commercial and non-commercial entities, to reduce risks associated with IP matters. IP risks have gained in importance as IP awareness grew. Recent and well publicized court actions have demonstrated the impact that IP can have on the missions of public and private institutions. Patent assertion entities, so-called patent trolls, have also been in the news. These entities seek to maximize the value of their patent portfolio by forcing other entities to enter in licensing deals with them by menacing them of legal actions. This is but a small fraction of all IP risks that can be incurred by organizations. Sound management of its own IP portfolio and sound planning of IP strategy are the most powerful tools to limit IP risks. This limited IP risk will in turn allow both CSA and its partners to focus on furthering their goals instead of dealing with IP-related legal issues.

Together, Oc1 and Oc2 also allow both the CSA and its partners, both commercial and non-commercial entities, to maintain the continuum of the innovation chain. The innovation chain refers to the concept of knowing what was developed, by whom and when, which will, in turn, allow transactions (contracts, collaboration agreements, publications, etc.) to be made with the technology in question. The IP officers establish the innovation chain of technology used or developed by the Crown. When technology is being developed by multiple entities, as is nearly always the case for space technologies, continuum of the innovation chain becomes complex. Indeed it is important to know who owns and or controls which part of the technologies developed. This in turn, allows each member of the chain of innovation to know what it can do with its part of the chain and who to contact to do more than what is already conceded by its contribution to the chain of innovation. For example this allows companies to seek permission of other partners to further develop technologies and create new products and services for the benefit of Canadians. This also allows universities to identify underdeveloped links and conduct further research that can benefit the whole space sector. Successful commercial and scientific use of space innovation can only be done when the continuum of the innovation chain is maintained.

Qualitative indicators have been selected that will demonstrated how the risks are reduced and how the continuum of the innovation chain is maintained. For example, giving a number of how many successful management of IP protection has been done in a year, does not accurately render how the risks were reduced. The intent is to demonstrate, through selected cases, how the risk reduction has been achieved. The same methodology will be used for managing the IP title chain (innovation chain).

Ultimate Outcome
Facilitates the circulation of space technologies and products for space and terrestrial applications for the benefit of Canadians (Oc4)

The effective circulation of IP allows the government to make its own IP available for the benefits of Canadians and facilitates the sharing of IP between all the players of the space sector for the benefits of all Canadians. Circulation of IP allows industry to derive new products from R&D activities. It allows academia to further the knowledge associated with space activities. It also allows the government to accomplish its mission by ensuring all the adequate rights are obtained. Moreover, good circulation of IP allows the government to demonstrate the contribution of the Canadian space sector to the global goals of space exploration and peaceful uses by allowing it to share the results of the Canadian space sector activities with everyone on Earth. The IPMTT facilitates the circulation of space technologies and products for space and terrestrial applications for the benefit of Canadians by following closely the portfolio established and taking appropriate actions to either protect CSA's IP or to make it available, when the opportunity arises.

For example, the IPMTT negotiated the necessary rights to allow Commander Chris Hadfield to cover David Bowie's Space Oddity song while he was in orbit. He thus became the first person to record a music video clip in space and he inspired millions of people across the world.

Appendix C: Bibliography

  • Canadian Space Agency Act, S.C. 1990, c. 13.
  • Canadian Space Agency. (2011). Evaluation of the STDP for the period of 2002/03 to 2007/08. St-Hubert.
  • Canadian Space Agency. (2011). Policy on the Management of Intellectual Property and Technology Transfer. St-Hubert.
  • Canadian Space Agency. (2014). Canada's Space Policy Framework: Launching the Next Generation. St-Hubert.
  • Canadian Space Agency. (2014). Evaluation of the Advanced Exploration Technology Development Program. St-Hubert.
  • Canadian Space Agency. (2015). Departmental Evaluation Plan 2015-16 to 2019-20. St-Hubert
  • Canadian Space Agency. (2015). Evaluation of the International Market Access Program (Comprising the European Space Agency Contribution Program) of the Canadian Space Agency. St-Hubert.
  • Canadian Space Agency. (2015). Management Action Plans Follow-up Evaluation: Annual Report as of March 31st, 2015. St-Hubert.
  • Canadian Space Agency. (2015). Report on Plans and Priorities 2015-2016. St-Hubert.
  • Government Consulting Services. (2011). Evaluation of the STDP for the period 2002-2003 to 2007-2008. Ottawa.
  • Government of Canada. (2010). Budget 2010: Leading the way in jobs and growth. Ottawa.
  • Government of Canada. (2011). The Budget Plan (June 6). Ottawa.
  • Government of Canada. (2014). Seizing Canada's Moment: Moving Forward in Science, Technology and Innovation 2014. Ottawa.
  • Industry Canada. (2011). Innovation Canada: A Call to Action. Ottawa.
  • Industry Canada. (2012). Reaching Higher: Canada's Interest and Future in Space, Vol. 2. Ottawa.
  • Industry Canada. (2012). Space Working Group Report. Ottawa.
  • Review of Federal Support to Research and Development, retrieved from rd-review.ca/eic/site/033.nsf/eng/home.
  • Treasury Board of Canada. Policy on Transfer Payments. Retrieved from: www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=13525&section=HTML.

Footnotes

Footnote 1

Section 4 of the Canadian Space Agency Act, S.C. 1990, c. 13.

Return to footnote 1 referrer

Footnote 2

Industry Canada. (2012). Space Working Group Report. Ottawa, p. 22.

Return to first footnote 2 referrer

Footnote 3

Industry Canada. (2012). Reaching Higher: Canada's Interest and Future in Space, Vol. 2. Ottawa, p. 1.

Return to footnote 3 referrer

Footnote 4

Canadian Space Agency. (2014). Canada's Space Policy Framework: Launching the Next Generation. St-Hubert, p. 8.

Return to footnote 4 referrer

Footnote 5

Industry Canada. (2012). Space Working Group Report. Ottawa, p. 37 and 38.

Return to first footnote 5 referrer

Footnote 6

For the purpose of this report, the notion of capability building refers to an organization's capability to develop and operationalize new space technology.

Return to footnote 6 referrer

Footnote 7

Industry Canada. (2012). Reaching Higher: Canada's Interest and Future in Space, Vol. 2. Ottawa, p. 1.

Return to footnote 7 referrer

Footnote 8

Unless otherwise stated, the description included in this sub-section is based on the information contained in the Performance Measurement Strategy developed by the STDP, dated October 2015.

Return to footnote 8 referrer

Footnote 9

For the purpose of this report, contributions systematically refer to "non-refundable contributions," which are defined as "any pre-commercial technology activities that are carried out to resolve unknowns regarding the feasibility of space concepts or applications in the space sector." See Section 5.2 of the June 2014 announcement of opportunities, available at: asc-csa.gc.ca/eng/ao/2014-stdp-ao2.asp.

Return to footnote 9 referrer

Footnote 10

See the mandate and objectives of the STDP (January 30, 2016). Retrieved from www.asc-csa.gc.ca/eng/programs/stdp/mandate.asp.

Return to footnote 10 referrer

Footnote 11

Canadian Space Agency. (2014). Canada's Space Policy Framework: Launching the Next Generation. St-Hubert.

Return to first footnote 11 referrer

Footnote 12

Due to rounding, amounts may not sum exactly to totals.

Return to footnote 12 referrer

Footnote 13

Contributions were only offered starting in 2012-13.

Return to footnote 13 referrer

Footnote 14

Salaries exclude Employee Benefit Plan.

Return to footnote 14 referrer

Footnote 15

O&M expenditures also exclude capital expenditures, which totaled $179,958 for the five years covered.

Return to footnote 15 referrer

Footnote 16

See Canadian Space Agency. (2011). Evaluation of the STDP for the period 2002-2003 to 2007-2008. St-Hubert.

Return to footnote 16 referrer

Footnote 17

Unless otherwise stated, the description included in this sub-section is based on the information contained in the Performance Measurement Strategy developed by the IPMTT group, dated October 2015.

Return to footnote 17 referrer

Footnote 18

Salaries exclude Employee Benefit Plan.

Return to footnote 18 referrer

Footnote 19

Canadian Space Agency. (2011). Evaluation of the STDP for the period of 2002/03 to 2007/08. St-Hubert.

Return to footnote 19 referrer

Footnote 20

See Canadian Space Agency. (2015). Management Action Plans Follow-up Evaluation: Annual Report as of March 31st, 2015. St-Hubert, p. 4.

Return to footnote 20 referrer

Footnote 21

Canadian Space Agency. (2015). Departmental Evaluation Plan 2015-16 to 2019-20. St-Hubert.

Return to footnote 21 referrer

Footnote 22

Unless otherwise stated, R&D projects or projects refer to technology development projects that are funded through an STDP research contract or contribution agreement.

Return to footnote 22 referrer

Footnote 23

Government of Canada. (2010). Budget 2010: Leading the way in jobs and growth. Ottawa, p. 83 and 246.

Return to footnote 23 referrer

Footnote 24

Review of Federal Support to Research and Development, retrieved from rd-review.ca/eic/site/033.nsf/eng/home.

Return to footnote 24 referrer

Footnote 25

Industry Canada. (2011). Innovation Canada: A Call to Action. Ottawa, p. 7-3.

Return to footnote 25 referrer

Footnote 26

Government of Canada. (2011). The Budget Plan (June 6), ch. 4.1.

Return to footnote 26 referrer

Footnote 27

Industry Canada. (2012). Reaching Higher: Canada's Interest and Future in Space, Vol. 2. Ottawa, p. 1.

Return to footnote 27 referrer

Footnote 28

Industry Canada. (2012). Reaching Higher: Canada's Interest and Future in Space, Vol. 2. Ottawa, p. 26.

Return to first footnote 28 referrer

Footnote 29

Industry Canada. (2012). Reaching Higher: Canada's Interest and Future in Space, Vol. 2. Ottawa, p. 2.

Return to footnote 29 referrer

Footnote 30

Industry Canada. (2012). Reaching Higher: Canada's Interest and Future in Space, Vol. 2. Ottawa, p. 5.

Return to footnote 30 referrer

Footnote 31

Government of Canada. (2014). Canada's Space Policy Framework: Launching the Next Generation. Ottawa.

Return to footnote 31 referrer

Footnote 32

Government of Canada. (2014). Canada's Space Policy Framework: Launching the Next Generation. Ottawa, p. 11.

Return to footnote 32 referrer

Footnote 33

Government of Canada. (2014). Seizing Canada's Moment: Moving Forward in Science, Technology and Innovation 2014. Ottawa.

Return to footnote 33 referrer

Footnote 34

Government of Canada. (2014). Seizing Canada's Moment: Moving Forward in Science, Technology and Innovation 2014. Ottawa, p. 11.

Return to footnote 34 referrer

Footnote 35

The decision to end almost all internal R&D activities has reduced the creation of new CSA-owned patents and trade secrets. However, other types of IP, such as official market or copyrights, were not affected by this decision.

Return to footnote 35 referrer

Footnote 36

For further discussion, see Industry Canada. (2012). Reaching Higher: Canada's Interest and Future in Space, Vol. 2. Ottawa, p. 26.

Return to footnote 36 referrer

Footnote 37

Canadian Space Agency. (2014). Canada's Space Policy Framework: Launching the Next Generation. St-Hubert, p. 9.

Return to footnote 37 referrer

Footnote 38

Industry Canada. (2012). Reaching Higher: Canada's Interest and Future in Space, Vol. 2. Ottawa, p. 5.

Return to footnote 38 referrer

Footnote 39

Canadian Space Agency Act (S.C. 1990, c. 13), s. 4.

Return to footnote 39 referrer

Footnote 40

Industry Canada. (2012). Space Working Group Report. Ottawa, p. 39.

Return to footnote 40 referrer

Footnote 41

Industry Canada. (2012). Space Working Group Report. Ottawa, p. 40.

Return to footnote 41 referrer

Footnote 42

See for instance: Canadian Space Agency. (2015). Evaluation of the International Market Access Program (Comprising the European Space Agency Contribution Program) of the Canadian Space Agency. St-Hubert.

Return to footnote 42 referrer

Footnote 43

For instance, the Flights and Fieldwork for the Advancement of Science and Technology (FAST) program.

Return to footnote 43 referrer

Footnote 44

For instance, the Earth Observation Application Development Program (EOADP) or the Government Related Initiatives Program (GRIP).

Return to footnote 44 referrer

Footnote 45

For instance, the CSA's stratospheric balloon program (STRATOS).

Return to first footnote 45 referrer

Footnote 46

For instance, the Cooperation Agreement between Canada and the European Space Agency, or the Industrial Research Chair (IRC) led by the Natural Sciences and Engineering Research Council of Canada (NSERC).

Return to footnote 46 referrer

Footnote 47

Canadian Space Agency. (2014). Evaluation of the Advanced Exploration Technology Development Program. St-Hubert.

Return to footnote 47 referrer

Footnote 48

As stated by the NASA, a technology that reaches the TRL 6 (out of a maximum of 9) is expected to be a fully functional prototype or representational model. See: www.nasa.gov/directorates/heo/scan/engineering/technology/txt_accordion1.html.

Return to footnote 48 referrer

Footnote 49

Canadian Space Technology. (2013). RFP for STDP-06, p. 4.

Return to footnote 49 referrer

Footnote 50

Canadian Space Agency. (2011). RFP for STDP-04, p. 2-3.

Return to footnote 50 referrer

Footnote 51

Treasury Board of Canada. Policy on Transfer Payments. Retrieved from: www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=13525&section=HTML

Return to footnote 51 referrer

Footnote 52

Canadian Space Agency. (2015). Report on Plans and Priorities 2015-2016. St-Hubert, p. 55.

Return to footnote 52 referrer

Footnote 53

Canadian Space Agency. (2011). Policy on the Management of Intellectual Property and Technology Transfer. St-Hubert.

Return to footnote 53 referrer

Footnote 54

Canadian Space Agency. (2015). IPMTT Performance Measurement Strategy. St-Hubert.

Return to first footnote 54 referrer

Footnote 55

Directed contracts are used by the CSA in instances where only one provider can technically offer the service required by the CSA.

Return to footnote 55 referrer

Footnote 56

The number of R&D projects and the value of contracts and agreements included in the table are those that relate to contracts and agreements that were still active at the time of the evaluation, and not necessarily the totals for each stream listed.

Return to footnote 56 referrer

Footnote 57

Outcomes from the 2009-2012 STDP internal client surveys related to 12 generic technology development research contracts and from the 2013-2015 STDP internal client surveys related to 48 mission-enabling technology development research contracts.

Return to footnote 57 referrer

Footnote 58

The expected reporting on contributions cover, among other things, the project's achievements and successes, research team and partnerships, public dissemination activities, and intellectual properties.

Return to footnote 58 referrer

Footnote 59

Canadian Space Agency. (2015). STDP Performance Measurement Strategy. St-Hubert.

Return to footnote 59 referrer

Footnote 60

Canadian Space Agency. (2015). IPMTT Performance Measurement Strategy. St-Hubert.

Return to footnote 60 referrer