Evaluation of the CASSIOPE Contribution Program

Download the PDF version (1.5 MB)

For the period from December 2003 to December 2013

Project # 13/14 02-01

Prepared by the Audit and Evaluation Directorate

February 2014

Table of Contents

  1. Introduction
  2. Background
  3. Evaluation Approach and Methods
  4. Results
  5. Conclusions and Recommendations

List of Tables and Figures

Abbreviations Used in the Report

  • AIT: Assembly, integration and testing
  • ARLU: Annual Reference Level Update
  • CASSIOPE: CAscade, SmallSat and IOnospheric Polar Explorer
  • CDS: Cascade Data Services
  • CRC: Communications Research Centre Canada
  • CSA: Canadian Space Agency
  • DND: Department of National Defence
  • ECG: Evaluation Consultative Group
  • EFI: Electric Field Instrument
  • ePOP: Enhanced polar outflow probe
  • ECG: Evaluation Consultative Group
  • ESA: European Space Agency
  • IC-TPC: Industry Canada – Technology Partnership Canada
  • MDA: MacDonald, Dettwiler and Associates Ltd.
  • NRCan: Natural Resources Canada
  • PCW: Polar Communication and Weather mission
  • RCM: RADARSAT Constellation Mission
  • Smallsat: Small satellite
  • TB: Treasury Board
  • U of C: University of Calgary

Executive Summary

Background

The CASSIOPEFootnote 1 Contribution Program was authorized for a five-year period on October 30, 2003. The project was extended due to the launch of the satellite being postponed several times. The launch took place in September 2013. The CASSIOPE mission combines three program elements: it provides for the integration of two payloads, the Cascade telecommunications demonstration payload and the Enhanced Polar Outflow Probe (ePOP) scientific payload, on a single generic Canadian small satellite (smallsat) bus.

The Canadian Space Agency (CSA) established two contribution agreements. One was with MacDonald, Dettwiler and Associates Ltd. (MDA) and its subsidiary, Cascade Data Services Inc., for the overall CASSIOPE mission technology demonstration phase, which included the Cascade payload, integration of the ePOP payload, and phases C and D (development and manufacturing) of the smallsat bus. The other was with the University of Calgary for the ePOP payload.

Total funding of the CASSIOPE project was $75 million (plus $6 million for internal CSA costs). In addition, Industry Canada's Technology Partnerships Program provided an additional $48.6 million in funding to MDA via a separate contribution agreement.

The evaluation of the CASSIOPE contribution program on behalf of the CSA Audit and Evaluation Directorate was undertaken by Kelly Sears Consulting Group in partnership with Beechwood Consulting and Research Inc., BBMD Consulting Inc., and Hickling Arthurs Low Corp.

Conclusions and Recommendations

Relevance

Missions such as CASSIOPE serve important commercial, scientific and societal needs of Canadians. They support the design and manufacture of world-class innovative technologies, products and services and provide training opportunities for young scientists and engineers. The objectives of the CASSIOPE project – design and manufacture of world-class technologies, training for young scientists and engineers, and promoting international partnerships – are well aligned with federal S&T priorities and the recently released Space Policy Framework. The Aerospace Review (i.e., the Emerson Report) published in November 2012 provides a solid rationale for federal support for the development of new satellites. The report explains that satellites are becoming more essential to modern economies and national security. Canada – with its vast geography, dispersed population, isolated communities, rich endowment of natural resources and northern location – has a particular need for space assets and applications. This rationale is echoed in the Space Policy Framework. In addition, the influence of space weather is of particular concern for Canada, particularly because the North Magnetic Pole is located just beyond the Canadian Arctic. This underscores the importance of research related to space weather, supported by the ePOP component of CASSIOPE.

Capacity

CASSIOPE has had a positive impact on the capacity of Canada's space industry. Specifically, key informants stated that the project contributed directly to maintaining capacity at the prime contractor and its subcontractors and may have helped broaden their product lines. However, no quantitative data on trends in employment due to the project in the various companies has been collected by the CSA to further support this finding.

Domestically, Canadian firms are highly dependent on the CSA and other federal departments such as the Department of National Defence in order to maintain their capacity. The general consensus is that Canadian space companies tend to experience cycles of "feast or famine." This makes it difficult for companies to maintain their capacity.

Recommendation #1: The CSA should make efforts to track the impacts of its projects on industrial capacity. The performance measures should be relatively easy for industry partners to collect data and report on.

Recommendation #2: The CSA should explore ways to better ensure the Canadian space industry has the technological capacity to undertake space projects sponsored by the agency. This would align with the Space Policy Framework and the Aerospace Review, providing firms in the Canadian space sector and academia with direction on anticipated work and facilitating planning and capacity development, allowing the industry and academia to better respond to the needs of the CSA.

Technology and Business Risks

The technology risks associated with CASSIOPE have been fully retired. A final commissioning review was completed on February 24, the spacecraft was found to be ready for routine operations, and the Cascade technology was successfully demonstrated.

With respect to the business risks, some feel the technology has become somewhat obsolete over the past 10 years with the emergence of fibre optic technology, putting in doubt whether CASSIOPE can ever be competitive. However, others argue that the Cascade technology was never intended to compete with fibre optic technology, stating that it was intended to efficiently transfer very large files from remote locations, ships and oil/gas rigs which are not served by fibre optics.

Scientific Research

The ePOP component of CASSIOPE is functioning as intended, and early data has been received enthusiastically by researchers. The ePOP data is seen as complementary to data being generated by the Swarm trio of satellites launched by the European Space Agency shortly after CASSIOPE in November 2013. There is optimism that ePOP data will make an important contribution over the coming few years (as long as CASSIOPE is in orbit) to the modelling of space weather and thus help enhance the security of radio transmissions, pipelines and electrical transmissions.

Design

Although the governance structure for CASSIOPE has worked well, the CSA did not exercise sufficient oversight of the repayment aspect of the MDA contribution agreement. CASSIOPE is the first Canadian satellite project that has used a repayable contribution mechanism.

Recommendation #3: If the CSA intends to use repayable contributions in future space projects, it should ensure that it has the internal capacity to both assess the merits of the business case presented by the recipient and to manage the repayment process.

Economy and Efficiency

Total funding for CASSIOPE was $123.6 million with $12 million for ePOP and $111.6 million for the Cascade and smallsat bus components. With the exception of an additional $1 million for ePOP, the value of the contribution agreements did not change over the 10-year period. It is very rare for space projects of the complexity of CASSIOPE to remain within budget. The use of a contribution agreement meant that industry had to absorb any cost overruns.

1. Introduction

The Canadian Space Agency (CSA) CASSIOPEFootnote 2 Contribution Program was authorized by Treasury Board for a five-year period on October 30, 2003. The project was extended due to the launch of the satellite being postponed several times. The launch took place in September 2013. The CASSIOPE program combines three project elements: it provides for the integration of two payloads, the Cascade telecommunications demonstration payload and the Enhanced-Polar Outflow Probe (ePOP) scientific payload, on a single generic Canadian Small Satellite (Smallsat) bus.

Two contribution agreements with recipients were developed by the CSA: one with MacDonald, Dettwiler and Associates Ltd. (MDA)/Cascade Data Services Inc. for the overall CASSIOPE mission technology demonstration phase (CX), including the Cascade payload, integration of the ePOP payload and phases C and D (development and manufacturing) of the smallsat bus; and the other with the University of Calgary for the ePOP payload. Total funding of the CSA CASSIOPE Contribution Program is $81 million. The contribution agreement called for repayment of the CSA's contribution by MDA/Cascade Data Services based on a 5% royalty rate on future Cascade service revenues, up to $63 million.

In addition, Industry Canada's Technology Partnership Program provided, through a separate contribution agreement, $48.6 million in funding to MDA, the prime contractor.Footnote 3 This funding also supported the entire CX phase as well as a portion of the pre-production phase of the subsequent Cascade satellites.

The evaluation of the CASSIOPE Contribution Program was conducted on behalf of the CSA Audit and Evaluation Directorate by Kelly Sears Consulting Group in collaboration with Beechwood Consulting and Research Inc., BBMD Consulting Inc., and Hickling Arthurs Low Corp. The evaluation is a requirement of the CSA five-year evaluation plan and was conducted in accordance with the Treasury Board of Canada Secretariat's Policy on Evaluation (2009). According to the CSA Evaluation Plan, the evaluation was to be completed by March 31, 2014. The evaluation was conducted by Kelly Sears between July 2013 and February 2014.

The evaluation covers the entire history of CASSIOPE, from the signing of the two contribution agreements in late 2003 to the end of December 2013. While the evaluation focuses on the CSA contribution program, information pertaining to the Industry Canada contribution agreement is included in the report.

A mid-term review of CASSIOPE was completed by Audit Services Canada in collaboration with the CSA in May 2007. Reference is made in the present evaluation to findings and conclusions from the previous review.

This report presents the findings and recommendations resulting from the evaluation. A description of the program, including the context, resources allocated, governance and logic model, is provided in Chapter 2. The evaluation approach and methodology is presented in Chapter 3. Findings for the issues of relevance and performance are presented in Chapter 4. Finally, overall conclusions and recommendations are presented in Chapter 5.

2. Background

2.1 Program Profile

The CASSIOPE Contribution Program resulted from a confluence of three initiatives, each of which started to gather momentum in the early 2000s. First, the Cascade satellite communications project (secure digital store-and-forward file delivery service) was being promoted by MacDonald, Dettwiler & Associates (MDA) and its wholly owned subsidiary Cascade Data Services (CDS). This initiative was already partly funded by Industry Canada's (IC) Technology Partnership Canada (TPC) ProgramFootnote 4 and was being considered for additional support by the CSA to mitigate significant technology risk. Second, the CSA was considering supporting the development of a multi-mission small satellite (smallsat) bus. Third, the agency was also considering a new science mission directed at launching a suite of eight scientific instruments on a microsatellite to investigate atmospheric and plasma flows and related wave-particle interaction processes in the ionosphere (enhanced Polar Outflow Probe (ePOP)) being put forward by the University of Calgary.

By 2003, discussions among all the principals involved in these initiatives concluded that it would be more efficient and likely equally effective if these three initiatives were combined into one CSA program, while leveraging the initial investment by IC-TPC in Cascade.

The CSA signed a contribution agreement in December 2003 amounting to $63 million with MDA for the development, deployment, and on-orbit demonstration of a smallsat bus and Cascade digital courier technology. The Cascade technology is intended to allow very large amounts of information to be delivered to anywhere in the world. The successful demonstration of the technology is expected to pave the way for a commercial "digital package delivery service" to users ranging from oil and gas exploration companies to medical facilities in isolated communities.

The CSA signed a contribution agreement with the University of Calgary (U of C) for ePOP in February 2004 for $11 million. The contribution agreement was subsequently amended in September 2008 to add $1 million, for a total of $12 million. The contribution agreement includes the development of eight scientific instruments, their integration in one payload, support to the integration in the satellite, and the operation of the science payload for one year after the commissioning in space of the satellite/payload.

The ePOP science payload is a suite of eight science instruments or experiments designed to help scientists expand their knowledge of near-Earth space. At higher and higher altitudes, the Earth's atmosphere becomes less dense and more ionized, merging into what is called the ionosphere and eventually into the region of space extending well past lunar orbit, called the magnetosphere. Many processes that are driven by the sun produce the aurora, affect satellites, and even affect distribution systems on the ground, such as power grids, navigation systems and pipelines. ePOP aims to help enhance understanding of these observations of particles, electromagnetic fields and auroral displays.

In June 2012, the IC-ITO formally de-scoped the pre-production phase involving the construction of up to four operational satellites. The de-scoping was initiated by Industry Canada because it was felt that CDS/MDA was not making progress on the commercialization aspects of the project and had thus far failed to secure an anchor customer for Cascade. As a result, CDS/MDA is no longer obligated to pursue the Cascade project as it was originally envisioned. As such, it is possible that the objective of building several Cascade spacecrafts in the future will not be met. In addition, the Early Entry Stage has been removed from the IC-ITO contribution agreement and thus no objectives related to this phase will be met. The de-scoping of the IC-ITO contribution agreement and its linkage to the CSA's CASSIOPE Contribution Program and specifically its contribution agreement with MDA means that the outcomes related to the commercialization of the project may not be achieved (i.e., establishment of the viability of the Cascade technology with prospective users, attraction of outside investment needed to build four more Cascade satellites and the creation of a new export-oriented global satellite-based courier service).

2.2 Governance, Roles and Responsibilities

Within the CSA organization (prior to a reorganization of 2010), the sponsoring branches were Space Science for ePOP and Space Technologies for the Cascade-payload and the small satellite bus. These branches reviewed the proposals put forward by MDA and the U of C and found the requirements compliant with the strategic objectives of the CSA leading to the provision by the CSA of the funding necessary for the management of the contributions. The CSA Space Utilization Branch is currently responsible for the management of the CSA CASSIOPE Contribution Program. However, after the project completion date (currently planned for December 2014), the experimental phase of the project will come to an end and no technical oversight will be required.

The Project Leader is the Director General (Space Utilization) who is responsible for establishing the management framework and the appropriate performance measurement, playing a coordination role with sponsored organizations (e.g., CDS/MDA, University of Calgary), providing the inputs required for the preparation of the CSA's annual Report on Plan and Priorities and the Departmental Performance Report, and approving payments of claims (or delegating them).

The Program Manager is responsible for establishing the organizational structure appropriate for managing the Contribution Program, assigning roles and responsibilities, implementing data collection measurement systems for reporting on progress and performance, monitoring risks and proposing mitigation measures, monitoring specific milestones and recommending payments of claims to the Project Leader with the appropriate documentation, and making information available to support activities such as internal audits. In the case of activities on the Cascade payload, the Program Manager is assisted by the Communications Research Centre, which acted as technical experts in satellite telecommunications for both the CSA and IC-ITO.

After the project completion date and until the contribution program end date (which would coincide with the repayment end date currently scheduled for 2033), the CSA Centre of Expertise in Grants and Contribution, in the CSA Space Science and Technology Branch, will manage the CASSIOPE Contribution Program. During that period, most of the activities will consist of tracking financial information and ensuring that repayments are compliant with the agreement.

2.3 Resource Allocation

Total funding to the CSA's CASSIOPE Contribution Program is $75 million with $63 million for the Cascade and smallsat bus components, and $12 million for ePOP, including the 2008 amendment which increased funding for ePOP. Based on budget information from the CSA, financial resources spent by the CSA are within budget, i.e., thus far there is no cost overrun.

Tables 2.1, 2.2 and 2.3 summarize the financial resources and FTEs for Cascade, ePOP, and the total CASSIOPE project, respectively, for 2003-04 to 2012-13.

Table 2.1: Forecast and Actual Expenditures ($) and Full-time Equivalents (FTEs) for Cascade
  2003-
04
2004-
05
2005-
06
2006-
07
2007-
08
2008-
09
2009-
10
2010-
11
2011-
12
2012-
13
Forecast budget ($) based on ARLU (contribution) - 14,300,000 25,000,000 18,000,000 8,500,000 6,000,000 2,720,000 570,000 250,000 -
Actual expenditures ($) (contribution) 5,700,000 14,300,000 14,500,000 16,180,000 6,000,000 3,600,000 1,900,000 570,000 250,000 -
Forecast CSA internal costs (operations, administration) based on ARLU - 5,700,000 1,386,400 1,185,410 1,103,245 253,400 240,840 72,870 69,240 71,736
Actual CSA internal costs (operations, administration) - 822,734 1,075,401 1,111,767 532,817 669,914 236,801 40,070 67,657 177,496
Forecast FTEs based on ARLU note 1 of Table 2.1 note 1 of Table 2.1 3.0 3.3 2.9 1.8 0.4 0.1 - 0.6
Actual FTEs 0 note 2 of Table 2.1 3.6 3.2 3.0 3.0 0.9 0.2 0.5 1.2

Table 2.1 notes

Table 2.1 note 1

Forecast FTEs 2.5 for CASSIOPE mission, no breakdown available for ePOP for 2003-04 and 2004-05.

Return to first table 2.1 note 1 referrer

Table 2.1 note 2

Actual FTEs 4.3 for CASSIOPE mission, no breakdown available for ePOP for 2004-05.

Return to table 2.1 note 2 referrer

Table 2.2: Forecast and Actual Expenditures ($) and Full-time Equivalents (FTEs) for ePOP
  2003-
04
2004-
05
2005-
06
2006-
07
2007-
08
2008-
09
2009-
10
2010-
11
2011-
12
2012-
13
Forecast budget ($) based on ARLU (contribution) - 3,200,000 2,950,000 1,781,000 1,168,000 700,000 807,000 300,000 460,000 -
Actual expenditures ($) (contribution) 360,000 3,150,000 3,234,000 2,281,000 1,668,000 500,000 207,000 140,000 60,000 150,000
Forecast CSA internal costs (operations, administration) based on ARLU - 272,000 968,400 353,770 426,275 712,125 533,930 598,930 428,060 248,216
Actual CSA internal costs (operations, administration) 317,759 253,356 247,512 279,483 358,544 492,576 544,160 603,940 576,359 360,574
Forecast FTEs based on ARLU note 1 of Table 2.2 note 1 of Table 2.2 0.6 1.8 0.8 0.5 0.3 0.3 - 0.4
Actual FTEs 2.3 note 2 of Table 2.2 1.8 1.0 0.8 0.5 0.3 0.1 0.5 0.5

Table 2.2 notes

Table 2.2 note 1

Forecast FTEs 2.5 for CASSIOPE mission, no breakdown available for 2003-04 and 2004-05.

Return to first table 2.2 note 1 referrer

Table 2.2 note 2

Actual FTEs 4.3 for CASSIOPE mission, no breakdown available for 2004-05.

Return to table 2.2 note 2 referrer

Table 2.3: Forecast and Actual Expenditures ($) and Full-time Equivalents (FTEs) for CASSIOPE (incl. Cascade and ePOP)
  2003-
04
2004-
05
2005-
06
2006-
07
2007-
08
2008-
09
2009-
10
2010-
11
2011-
12
2012-
13
Forecast budget ($) based on ARLU (contribution) - 17,500,000 27,950,000 19,781,000 9,668,000 6,700,000 3,527,000 870,000 710,000 -
Actual expenditures ($) (contribution) 6,060,000 17,450,000 17,734,000 18,461,000 7,668,000 4,100,000 2,107,000 710,000 310,000 150,000
Forecast CSA internal costs (operations, administration) based on ARLU - 5,972,000 2,354,800 1,539,180 1,529,520 965,525 774,770 671,800 497,300 319,952
Actual CSA internal costs (operations, administration) 317,759 1,076,090 1,322,913 1,391,249 891,361 1,162,490 780,961 644,011 644,016 538,070
Forecast FTEs based on ARLU 2.5 2.5 3.6 5.1 3.7 2.3 0.7 0.4 - 1.0
Actual FTEs 2.3 4.3 5.4 4.2 3.7 3.5 1.1 0.4 1.0 1.7

2.4 Prior Audit and Review of the Program

This evaluation is not the first review of the CASSIOPE program: there was an audit and a mid-term review in 2007. The audit concluded that CSA management had adopted systems and procedures that made for appropriate monitoring of the program, particularly with respect to its technical aspects. However, the review also found that improvements were needed regarding the administrative aspect of claim payments. A management action plan was put in place following the recommendations.

The previous mid-term review of the CSA's CASSIOPE Contribution Program was approved in April 2009. The review found that the program's contribution agreements constituted an adequate funding vehicle and that effective management mechanisms had been implemented for the technical aspects of the program. However, the review also found that progress was insufficient toward meeting expected commercial results and securing subsequent repayment of the contribution. Recommendations pertaining to improved accountability, monitoring and reporting mechanisms were addressed by the CSA in a Management Action Plan, which was fully implemented by March 2010.

2.5 Program Theory

The logic model for CASSIOPE is presented in Figure 2.1 overleaf. The logic model narrative is provided in Appendix A.

Figure 2.1 CASSIOPE Contribution Program Logic Model

3. Evaluation Approach and Methods

3.1 Approach and Methods

3.1.1 Approach

In the parlance of evaluation research methodology, the design chosen for the CASSIOPE evaluation was a basic "post-test only non-experimental design," whereby the evaluation team observed the status of the CASSIOPE project at one point in time, a few months after launch of the satellite which took place in September 2013. As discussed later in this section, the timing of the evaluation meant that it was premature to fully assess the project's achievement of its intermediate and longer-term outcomes. In effect, the evaluation was similar to a mid-term evaluation, in that it assessed the performance of the project in producing each of the intended outputs and conducted a preliminary assessment of its success in achieving its outcomes.

The methodological approach and level of effort for this evaluation were determined using a risk-based approach. The evaluation relied on interviews both within the CSA and with recipients and other stakeholders together with a document and file review. The perspectives provided by both the CSA and external key informants were very consistent. While the original evaluation design called for additional interviews with users of the ePOP data, this was not possible given the recent launch of the satellite. In order to ensure accuracy of findings, interview summaries were validated by key informants from the various companies. The data collection methods are described further in section 3.1.2.

The evaluation team worked closely with an Evaluation Consultative Group (ECG). Members of the ECG included CSA staff and managers involved in the implementation of CASSIOPE as well as representatives from the Audit and Evaluation Directorate within the CSA. The ECG provided input and feedback on key deliverables for the evaluation, including project Workplan; Evaluation Plan; interview guides; presentation of preliminary findings; and the final report. The ECG also provided names of individuals to be interviewed.

Finally, the recommendations provided in this report are intended as lessons learned for improving the CSA's management of similar projects in the future.

3.1.2 Data Sources

3.1.2.1 Document Review

A review of documentation that relates to the CASSIOPE mission was undertaken to help address all of the evaluation issues. A large volume of documents were provided by the Project Authority and other members of the ECG and these documents were reviewed as part of the process of developing the Workplan, Evaluation Matrix and Evaluation Plan. Additional documents were identified by the research team through an internet search. All documents were reviewed systematically, using a template based on the evaluation matrix.

Although no challenges were encountered during the document review, it must be noted that a number of documents received were marked as confidential and could not be quoted. Although this provided important context to the evaluation team in interpreting the findings, the information contained in the documents could not, in many cases, be used in the report. This challenge was mitigated through the use of other documents and interviews.

A list of documents reviewed is included in Appendix B.

3.1.2.2 Key Informant Interviews

Key informant interviews served as an important source of qualitative information for this evaluation. Interviews were conducted with individuals representing the CSA, MDA, Magellan, the Communications Research Centre Canada (CRC), the University of Calgary and Industry Canada using tailored interview guides. The interviewees were identified by members of the Evaluation Consultative Committee and represented individuals who had been directly involved in the CASSIOPE project (i.e., a purposeful sample). Interviews were conducted both by phone and in-person.

A key limitation in the interview findings is the relatively small number of interviewees. Reasons for this include the nature of the subject being evaluated, i.e., a project as opposed to a program and the recent launch of CASSIOPE which meant that it was premature to interview potential users of the ePOP data, for example. The interviewees provided a variety of perspectives and included individuals with limited or no vested interest in the CASSIOPE project, thus affording an objective perspective of the mission.

With respect to analysis, the relatively small number of interviewees (n=20) and the specific roles and responsibilities of each interviewee means that reporting interview findings using counts of interviewees (i.e., "how many said what") is neither relevant nor appropriate due to issues of confidentiality.

Interview guides are included in Appendix C.

3.2 Purpose and Scope

The CSA Audit and Evaluation Directorate required the evaluation of the CASSIOPE Contribution Program as per the CSA's five-year departmental evaluation plan and in accordance with the Treasury Board of Canada Secretariat's Policy on Evaluation (2009). The Financial Administration Act (section 42.1) and the Directive on Transfer Payments (Appendix H, section 8), and the Policy on Evaluation (section 6.1.8) specify that all ongoing grants and contributions must be evaluated no less frequently than once every five years.

The approved CASSIOPE Results Based Management and Accountability Framework (RMAF) states that a summative evaluation is to be carried out following one year of CASSIOPE operations (originally planned to be late 2007-08). Due to factors outside of the CSA's control, the launch of CASSIOPE was considerably delayed, and finally occurred in September 2013.

The evaluation was designed to address the relevance of the objectives of CASSIOPE and the performance of the satellite in achieving them.

3.3 Evaluation Issues

The evaluation focuses on the five core issues identified in the Treasury Board of Canada Secretariat's Directive on the Evaluation Function (2009), which includes issues of relevance and performance (effectiveness, efficiency and economy). The Evaluation Matrix, outlining the evaluation issues and questions, indicators and data sources is presented in Appendix D. The questions addressed by the evaluation are listed below. Note that "CASSIOPE Contribution Program" and "CASSIOPE project" are used interchangeably throughout this report.

Relevance

  1. Is there a continued need for the CSA to be involved in investing in a project such as CASSIOPE?
  2. Is the CASSIOPE project aligned with federal government priorities?
  3. Is the CASSIOPE project consistent with federal roles and responsibilities?

Performance

  1. To what extent have CASSIOPE activities been implemented as intended?
  2. To what extent has CASSIOPE produced its expected outputs?
  3. To what extent has CASSIOPE achieved its immediate outcomes?
  4. To what extent has CASSIOPE achieved its intermediate outcomes?
  5. To what extent has CASSIOPE achieved its ultimate outcomes?
  6. To what extent has CASSIOPE contributed to the Government of Canada Whole-of-Government Framework?
  7. Is the project design appropriate for achieving expected program results?
  8. Have there been any unintended (positive or negative) outcomes?
  9. Is the project undertaking activities and producing outputs in the most efficient manner?
  10. Is the project achieving its intended outcomes in the most economical manner?

3.4 Limitations and Risk Mitigation

The main limitations and risks faced by the evaluation along with the mitigation strategies are summarized in the following table:

Table 3.1 Limitations to the Evaluation and Mitigation Strategies
Limitations/Risks Mitigation Strategies
Absence of baseline data – Because a performance measurement strategy had never been developed or implemented, no baseline data was available on many of the outcomes (e.g., number of HQP jobs created by the project). This is a problem common to many federal government evaluation studies. This issue mainly concerned assessing the impacts of CASSIOPE maintaining the core manufacturing capabilities of the Canadian satellite communications industry. The evaluation gathered primarily qualitative evidence from key informants on this issue. Interviews were undertaken with the prime contractor as well as several subcontractors in order to obtain a variety of perspectives.
Difficulties in measuring several outcomes – Since the CASSIOPE satellite was launched very recently, it has not been in operation long enough for many of the intended outcomes to have been achieved. The evaluation was able to assess whether most of the intended outputs had been achieved and most of the immediate outcomes. Some evidence was collected on the intermediate and longer-term outcomes. A full summative evaluation is scheduled in the future, at which time a more fulsome assessment of the outcomes would be possible.
Attribution of outcomes to the CASSIOPE project – The CASSIOPE project is one of many satellite missions/space projects being undertaken in Canada. Thus the assessment of attribution of change relative to some of the intended CASSIOPE project outcomes was expected to be difficult to attribute solely to the project. Attribution was not an issue for CASSIOPE's immediate outcomes. As noted above, due to the timing of the evaluation, it was premature to fully assess the intermediate and longer-term outcomes. Where findings are presented, the report is careful not to attribute change solely to CASSIOPE.
The subject matter is highly technical – There was a risk that the evaluation team might not understand some of the evidence collected (e.g., interviews of technical specialists) or develop incorrect findings. The evaluation team included consultants with science/engineering backgrounds and experience in evaluating space programs and projects. The evaluation team also worked very closely with CSA personnel to ensure the team gained a clear understanding of CASSIOPE's activities, outputs and outcomes.

4. Results

This chapter presents evaluation findings related to the relevance and performance of the CASSIOPE Contribution Program.

4.1 Relevance

The issue of relevance concerns whether or not a program continues to serve a demonstrable need, is responsive to the needs of Canadians and is an appropriate activity for the Government of Canada. Given that CASSIOPE is a one-time project, and not an ongoing program, the analysis took the perspective of examining the relevance of missions of a similar nature to CASSIOPE, i.e., projects that support the development of advanced systems intended to maintain Canadian space manufacturing capabilities and contribute to scientific knowledge and information of importance to Canada.

4.1.1 Continued Need for the Program

A sound rationale for CASSIOPE-type missions exists. This rationale reflects the objectives of CASSIOPE-type missions that serve commercial, societal and scientific needs.

4.1.1.1 Commercial, Societal and Scientific Needs

The Aerospace Review report published in November 2012 provides a solid rationale for the Government of Canada's investment in the development of new satellites.Footnote 5 The report explains that space is becoming ever more essential to modern economies and national security. Satellites are playing growing roles in fields as diverse as precision agriculture, resource extraction, meteorology and climatology, environmental monitoring, the delivery of education and health services, emergency response, border surveillance, the operation of civil and military drones, and the rapid deployment of armed forces. And it is not just big, expensive satellites that are providing such capabilities: smaller, cheaper satellites are becoming increasingly sophisticated, offering public and private sector customers a wider range of options when they buy and use space assets. Canada – with its vast geography, dispersed population, isolated communities, long coastlines, rich endowment of natural resources, and northern location – has a particular need for space assets and applications. This perspective is echoed in the just-released 2014 Space Policy Framework.

The CASSIOPE program contributes to Canada's societal and scientific needs by supporting the design and manufacture of world-class innovative technologies, products and services, by providing training opportunities to young scientists and engineers and by promoting Canadian industry abroad through exposure to international partnerships. In addition, several key informants emphasized the importance of the scientific needs served by CASSIOPE – specifically, the knowledge generated pertaining to space weather and the changing environmental conditions in near-Earth space. The roots of CASSIOPE's scientific mission reach back to Canada's first satellite, Alouette 1, launched in 1962. Over the past 50 years, space scientists – including leading academics working in several Canadian universities – have demonstrated a growing appreciation of this important boundary region and the ways it can influence the environment we live in.

The influence of space weather is of particular concern to Canada. For example, airline traffic over the poles is increasing, as circumpolar routes are more economical to operate. Route diversion due to space weather events would add to their operational costs. And with the North Magnetic Pole being situated just beyond Canada's Arctic territory, our country is particularly susceptible to the influences of space weather events.

Key informants emphasized that it is critically important for the Canadian scientific community to maintain its leadership role internationally in the study of space weather. Thus missions such as CASSIOPE and its ePOP scientific instruments are important to maintaining Canada's leadership role in this domain.

The Aerospace Review report emphasizes that satellites are a key instrument of government policy in the North, in order to support wealth creation, to protect the environment and to assert Canadian sovereignty. Given the intensification of multiple, conflicting claims in the Arctic, both international law and geopolitics demand that Canada be active in the region if it wants to secure its interests there.

The CASSIOPE mission also supports a segment of the Canadian space industry that is well positioned to compete internationally: small satellites. Smallsats are becoming more and more attractive to governments and private companies as a way of carrying out key activities in space with lower costs and shorter timelines compared to traditional larger satellites.

4.1.1.2 Need for financial support

Another aspect of need is whether financial support from the federal government is required to support the development of Canada's space industry. As noted in the Aerospace Review Report, a particular feature of progress in space in all countries has been the pervasive presence of government.Footnote 6 This is due to a variety of factors, including the reality that space is a "long game" with significant risks. The report does note that this reality is changing, as more and more private companies capitalize on space-related opportunities (such as SpaceX, the US company that launched the CASSIOPE satellite). But the report emphasizes one incontrovertible truth: space will be vital to securing Canada's national interests in the long term. The federal government clearly has an important role to play.

Canada's public space program has always involved commercial expertise and public-private sector collaboration. For example, the creation of Telesat and its current global role in providing commercial communication satellite services had its origins in experimental communication satellites developed through public-private sector collaboration as did the development of satellite data processing and applications using RADARSAT and other satellite data sources, now led by the private sector, to meet government and private sector mapping and surveying needs.

4.1.2 Alignment with Federal Priorities

The objectives of CASSIOPE – design and manufacture of world-class technologies, training for young scientists and engineers, and promoting international partnerships – are aligned with the recently released Space Policy Framework and the broader federal government S&T priorities set out in the Federal S&T Strategy.

The S&T StrategyFootnote 7 is guided by the following principles: promoting world class excellence, focusing on priorities and encouraging partnerships. The CASSIOPE project is well aligned with this strategy. For example, CASSIOPE is meeting the challenge of maintaining Canada's excellence in space research through the ePOP payload, which will produce scientific data on plasma in the atmosphere and increase the capabilities of space weather models. Improved prediction of space weather will increase the security of radio transmissions, pipelines and electrical transmission systems. This data will be shared with university centres across Canada and abroad, maintaining and developing new international partnerships.

The Space Policy Framework highlights five guiding principles which will inform Canada's space activities. These include putting Canadian sovereignty, security and prosperity first; supporting and utilizing the domestic space industry; fostering partnerships; focusing on excellence; and developing capacity. The CASSIOPE mission reflects the principles outlined in the Space Policy Framework through its focus on building Canadian space technological and management capacity particularly where Canadian firms have developed expertise, supporting excellence in space research, and developing partnerships with key space agencies in the U.S., Europe and Japan focusing on satellite technology. In addition, the CASSIOPE project represents a unique mission, bringing together the interests of industry, academia and government, providing a possible template for future collaborations.

CASSIOPE is also firmly aligned with the CSA's priorities. The mission's objectives are aligned with the CSA Strategic Outcome that states that Canada's exploration of space, provision of space services and development of its space capacity meets the nation's needs for scientific knowledge, innovation and information. Furthermore, the CASSIOPE's development and operation of a smallsat bus reflects one of the CSA's two program priorities: "develop and integrate small satellite advanced technology to provide fast and cost effective responses to government needs in specific areas such as safety and security, atmospheric monitoring, environmental stewardship, water quality monitoring and precision farming."

4.1.3 Alignment with Federal Roles and Responsibilities

There is no question that financial support for a CASSIOPE-type mission is firmly a federal government responsibility. The CSA has the legal mandate to support the Canadian space industry and the CSA is recognized as the government's lead organization. No other organization, such as a private satellite company, would launch, on its own, without government support, a CASSIOPE-type mission that featured a scientific payload as well as a high-risk payload intended to demonstrate the viability of a space-based digital courier service.

This aspect of the relevance issue is whether financial support from the CSA (i.e., via the CASSIOPE contribution program) to the Canadian space industry is a legitimate role for the federal government versus a role more appropriate for other levels of government or the private sector.

As noted in the Canadian Space Agency Act (last amended March 16, 2012), the Agency may exercise its powers, and perform its duties and functions, in relation to all matters concerning space over which Parliament has jurisdiction and that are not by, or pursuant to law, assigned to any other department, board or agency of the Government of Canada. And as noted in the Aerospace Review report, Canada's space program is led by the CSA.

The CSA's role in supporting the CASSIOPE mission is justified by the following factors:

  • The federal government plays a critical role in fostering innovation through funding for R&D. The CSA administers the main federal programs targeted to applied R&D in the Canadian space sector and to maintaining the excellence of Canada's space science. Given the significance of innovation to the long-term viability of the space sector, together with the scale of several other countries' support in space-related R&D, ongoing financial support by the CSA is critical.
  • The ePOP component of CASSIOPE is a scientific mission with no direct, immediate commercial benefits. The knowledge and information produced will be used by the academic/research community to improve space weather modelling. The eight scientific instruments have no commercial application. No private satellite company could generate revenues by launching a smallsat bus with only a scientific payload.
  • The Cascade component of CASSIOPE was high risk and the potential revenue stream from the following commercial phase "space-based digital courier service" was not expected for many years. In the absence of federal funding via the CSA and IC-TPC, it is highly unlikely that the CASSIOPE mission would have proceeded. Without mitigating the technical risks associated with the experimental phase of CASSIOPE via a contribution by the CSA and IC-TPC, a commercial mission sponsored by industry would not have gone ahead because there were no potential revenues expected from the experimental phase of CASSIOPE.
  • The demonstration through CASSIOPE of the feasibility of combining different payloads in a smallsat bus creates the potential for other CSA-sponsored missions that can take advantage of the cost efficiencies and other benefits of multiple payloads in a smallsat bus.

4.2 Performance

This section addresses the evaluation questions related to performance. These include questions related to implementation of activities, the production of outputs, achievement of outcomes and economy and efficiency. We focus this section on only those activities, outputs and outcomes that were not directly part of the IC-ITC de-scoping in 2012.

4.2.1 Implementation of Activities

Based on findings from the document review and key informant interviews, CASSIOPE was implemented largely as planned. Changes mainly to the cash flow and schedule were required as the project progressed and these were reflected in the numerous amendments to both contribution agreements. Although the delay in launching CASSIOPE was significant, this is not unusual for space projects given their complexity and, in the case of CASSIOPE, the need for coordination of three elements (ePOP, Cascade and the smallsat bus) made delays more likely. The use of SpaceX as the launcher also resulted in significant delays because this was a new launch company and thus more risky (albeit less expensive). The spacecraft was in storage for several years waiting for a launch opportunity to be realized.

The CASSIOPE program brought together three separate initiatives: MDA's Cascade secure digital file service delivery project; the CSA's development of a multi-mission smallsat bus; and ePOP. These activities have been implemented as originally planned but the time frame and funding have been amended three times. In February 2008, the funding for ePOP was increased by $1 million and completion of the program element was extended from October 31, 2008, to October 31, 2009. In 2009, the program was extended again to March 2012 and in February 2012 the program was finally extended to March 31, 2015, to account for delays in the launch. The terms and conditions of the CASSIOPE program are being amended to extend the repayment of the CSA's contribution to Cascade to 2033 (as noted earlier, repayment is based on a 5% royalty on Cascade service gross revenues, up to maximum repayment of $63 million).

The main challenges as identified by interviewees were the following:

  • Significant delays to the launch date. The original plan was for MDA, the prime contractor, to use a Russian launcher for CASSIOPE. However, due to a substantial price increase, MDA negotiated an agreement with a US company, SpaceX, when it was in its start-up phase. This resulted in numerous delays: the launch date in the contribution agreement was targeted for June 2008, while the actual launch took place in September 2013. Other issues which some interviewees believe contributed to delays in spacecraft delivery included under-estimating the complexity of the task resulting from combining the three mission objectives (small satellite bus development, ePOP and Cascade), and staff turnover at Magellan who built the smallsat bus. However others noted that although integration of the space craft components was more complex than anticipated and took longer, it did not cause the delays in the launch of CASSIOPE.
  • The launch delay led to a risk that the team (i.e., capacity) at the U of C would disperse and would not be available to operate the ePOP experiments and undertake data analysis. Partly to mitigate this risk, MDA transferred the day-to-day operations of the satellite to the U of C for a period of 18 months post-launch and after commissioning. This allowed for funding to be shifted from MDA to the U of C which provided sufficient cash flow for the U of C to keep the research team intact. This shift in responsibility and funding also allowed MDA to decrease its costs (i.e., it was cheaper for MDA to have the U of C operate ePOP than to handle it in-house).
  • It is generally believed that the delays led to problems with the ground terminal, which has made it difficult to get the ground terminal operational post-launch. There is a sense among interviewees from the CSA and MDA that the problems with the ground terminal function were directly attributable to the launch delays and CASSIOPE having been in storage for four years. Issues with the ground terminal have been resolved with commissioning completed in February 2014.
  • The use of a contribution agreement (instead of a contract). The use of a contribution agreement instead of a contract serves to shift the risks of cost overruns from the government to industry (the prime contractor and its subs). This decision was seen by a few interviewees within the CSA as a challenge to the agency's ability to manage the project because it did not provide the CSA with sufficient leverage to impose requirements on the prime contractor. For example, these interviewees stated that at the time, SpaceX was a new company with no proven track record, but the CSA had no influence on MDA's decision to work with SpaceX to launch CASSIOPE.
  • Perception that the budget for CASSIOPE was insufficient. This issue was raised by a few interviewees and was seen to have led to attempts to decrease costs which in turn increased the level of technical risk. For example, one interviewee noted that components that were commercial rather than space grade had to be used in order to keep within budget. The budget for a contribution agreement is fixed and this serves to shift the risk of cost overruns from government to industry. An insufficient budget will force industry to find ways to decrease costs and/or to absorb cost overruns.

4.2.2 Achievement of Expected Outputs and Outcomes

This section addresses the extent to which the expected outputs and outcomes for the CSA CASSIOPE Contribution Agreement have been achieved. The IC-ITO formally de-scoped the pre-production phase involving the construction of up to four operational satellites. This section does not consider expected outcomes that were part of this de-scoping.

4.2.2.1 Production of Expected Outputs

This section addresses whether the expected outputs for CASSIOPE have been delivered. The expected outputs were not affected by the de-scoping by IC-ITC in 2012. All of the expected outputs have been delivered.

4.2.2.1.1 Delivery of Components and Subsystems Required for Building the Cascade Payload

Based upon findings from the key informant interviews, this output has been achieved with the successful delivery of all components and sub-components required for the building of the Cascade payload. However, the document review revealed evidence of budget pressures on MDA which resulted largely from the launch delay. There were budget pressures resulting from a number of unexpected non-conformances which arose from the assembly integration test (AIT) performed prior to launch in September 2013 and which required updates or repairs.Footnote 8

4.2.2.1.2 Delivery of Components and Sub-systems Required for Building the Cascade Ground Terminals and Control Centre

According to findings from the final commissioning review, the ground terminal is operational.

The development of the ground terminal was subcontracted to an American firm which delivered the ground terminal in 2007. At that time, the ground terminal was tested and everything worked well. However, CASSIOPE was then put into storage at MDA-Montreal for over four years due to the launch delays. Shortly prior to launch in September 2013, MDA discovered that there were problems with the ground terminal. The American firm that built the ground terminal is no longer in that business and there is no follow-up service agreement between the American firm and MDA. At the time the key informant interviews were being completed (January 2014), there were efforts underway to resolve the issues with the ground terminal, with three separate teams working to find a solution. A solution was found and the commissioning review was successfully completed in February 2014.

The Experimental Service Control Centre has been delivered and is fully operational.

4.2.2.1.3 Design and Manufacture of a Smallsat Bus

The CASSIOPE smallsat bus was designed and built by Magellan Aerospace in Winnipeg. Based on interview findings and the document review, the bus was completed on schedule but was placed in storage by MDA dues to launch delays. According to CSA representatives, the bus meets all of the agency's requirements.

Based on interview findings, Magellan experienced some budget pressures because the price of the bus was not estimated to include the addition of the Cascade payload. However, this did not affect the cost of the project from the perspective of the CSA, although it did result in additional required investments on the part of Magellan.

4.2.2.1.4 Delivery of Instruments and Data Handling Unit Required for Building ePOP

Findings from the key informant interviews and document review indicate that all instruments and the data handling unit required for ePOP were successfully delivered. The only issues encountered were related to the payload being in storage for four years prior to launch. However these were resolved prior to the September 2013 launch.

4.2.2.1.5 On-Orbit Operations of the Cascade Payload and Demonstration of the Digital Satellite Courier Service Courier Service

The commissioning of Cascade payload was completed in February 2014 and the capabilities of the technology were successfully demonstrated.

4.2.2.1.6 On-Orbit Operations of the CASSIOPE Spacecraft

Based on findings from key informant interviews, this output has been achieved. There was agreement that the on-orbit operation of the spacecraft has been accomplished with the successful demonstration and official approval of the technology taking place on November 22, 2013 (i.e., the Commissioning Complete Review). One CSA representative noted that the Complete Review was done within 60 days of launch which is much faster than the typical 90 days experienced by other missions.

4.2.2.1.7 On-Orbit Operations of the ePOP Payload

According to the key informant interviews, this output has been delivered. Overall there is agreement that the ePOP payload is functioning as intended and gathering data. All eight instruments were reported by the U of C in mid-December 2013 to be fully operational.

4.2.2.2 Achievement of Immediate Outcomes

This section presents evaluation findings related to the achievement of immediate outcomes. Findings are based only on key informant interviews. Documentation reviewed did not contain information related to immediate outcomes because CASSIOPE has only recently been launched (September 2013).

4.2.2.2.1 Reduction of Technology and Business Risks Associated with Cascade

As noted previously, the Cascade technology has been successfuly demonstrated thereby eliminating the technology risks associated with Cascade. With respect to the business risks, there is consensus that the market for Cascade technology is different than that envisioned in 2003 when the project was implemented. Marketing of the Cascade technology by MDA has re-started but it remains to be seen whether a sufficient market for Cascade exists.

MDA representatives explained that the CASSIOPE project has contributed to a reduction in technology risks as well as reducing the business risks. Having the Cascade technology in space has made it more credible to potential clients from both the technological and business perspectives and MDA has restarted business development efforts for the Cascade technology. MDA representatives believe there is a market for the Cascade data service; however, potential customers are different from those envisioned when the CASSIOPE project was initiated back in 2003. MDA representatives believe that there may be some potential to generate revenue from the current CASSIOPE mission if it remains operational beyond 18 months. Looking beyond the CASSIOPE mission, MDA representatives believe that full implementation of the Cascade technology, with paying customers, will take about three years.

However not all individuals interviewed believe that there is sufficient market potential for Cascade. The Cascade data transfer technology is intended for transferring data between two points where one, or both, points are not fixed or they are in remote area. The emergence of fibre optic technology has, according to some interviewees, resulted in decreased market potential for the Cascade technology because fibre optic technology is accessible almost everywhere on Earth. However it should also be noted that fibre optics is not an economically viable technology in remote and sparsely populated areas or at sea.

Another potential limitation to decreasing the technology and business risks associated with the Cascade technology identified by CSA interviewees relates to the use of only a single point to demonstrate the technology rather than two points as was described in the original (2000) proposal to Industry Canada from MDA. There is a belief that demonstrating the transfer of data using a single point will not be sufficient to convince potential clients of the efficacy of the technology.

4.2.2.2.2 Improved Suitability of CASSIOPE Smallsat bus for Future Cascade Satellites

According to representatives from the CSA who were involved in CASSIOPE back in 2003, the smallsat bus developed by Magellan for the CASSIOPE mission was never intended to be used in subsequent Cascade missions. The smallsat bus is more sophisticated and larger (to accommodate a propellant tank) and thus more expensive than what would be required by MDA for future Cascade satellites because it was designed to support future CSA Earth observation missions. MDA would use a smaller, less sophisticated and thus less expensive bus specifically designed for Cascade.

There is agreement that, at present, there are no plans to build additional Cascade satellites because MDA has not attracted an anchor customer for Cascade. However, MDA plans to deploy Cascade as hosted payloads, i.e., on other satellites.

4.2.2.2.3 Increased Scientific Knowledge Resulting from ePOP

According to findings from the interviews, the ePOP component of CASSIOPE is successful and is very likely to contribute to the acquisition of scientific knowledge. According to interviewees, the commissioning of ePOP was successful with all eight instruments working and able to generate scientific data. After analyzing early data from ePOP, the U of C reported in mid-December 2013 that the quality was excellent with resolution of imagery being unprecedented. Preliminary results were presented to the US Geophysical Union in December and were very well received by the international scientific community in attendance.

Representatives from the U of C explained that the European Space Agency's (ESA) Swarm missionFootnote 9 and ePOP are producing complementary data on the effect of the ionosphere on the earth's magnetic field. The U of C led the development of the electric field instrument (EFI) on Swarm and each of the three satellites within Swarm carries a super thermal electron imager. Scientists working on ePOP continue to work collaboratively with European scientists in investigating the magnetic field.

4.2.2.3 Achievement of Intermediate Outcomes

This section addresses the extent to which CASSIOPE achieved its expected intermediate outcomes.

4.2.2.3.1 Maintenance of Core Manufacturing Capabilities of the Canadian Satellite Communication Industry

Based on interview findings it can be said that although a relatively small project, CASSIOPE contributed to maintaining and developing capacity within the Canadian satellite communications industry.

CASSIOPE contributed to mitigating a slowdown in business in 2003 for MDA in particular. According to interviewees, CASSIOPE had a direct impact on the ability of MDA and Magellan to keep core teams intact within both companies. When CASSIOPE was implemented in 2003, MDA-Montreal (then EMS) had just recently completed the RADARSAT-2 project and CASSIOPE allowed the company to maintain its existing staff and momentum. According to the IC-TPC contribution agreement amendment in 2012, the CASSIOPE Program has created and maintained over 300 person years of employment in the Canadian space industry.

Both MDA and Magellan were able to develop new capacity through the hiring of additional staff and developing new products and business lines as a direct result of CASSIOPE. For example, Magellan was able to maintain and hire new staff, many of whom are now working on the RADARSAT Constellation Mission (RCM). Magellan was also able to put in place a clean room for testing equipment. In the case of MDA, the company developed Ka-band expertise which has led to the development of new Ka products. However, there is a sense that the designs and components that were developed for CASSIOPE are now largely obsolete and cannot be used in future missions – the technology has evolved considerably over the past 10 years. For example, the chip sets have evolved and the circuits have been simplified over time. However, key informants believe that the knowledge could be put to use in future missions.

Despite the increased capacity which resulted from CASSIOPE, many interviewees question the long-term viability of the current capacity and manufacturing capability. For example, although Magellan has developed capacity in developing and building small satellite buses, it is unlikely that MDA will seek out Magellan as a subcontractor because MDA has recently purchased Loral, a US-based satellite bus manufacturer. In addition, the international market is also limited because potential customers tend to use the bus manufacturing capabilities in their own countries. Interviewees from the CSA noted that many Canadian firms are dependent on CSA contracts in order to survive. This issue is also noted in the Aerospace Review report. This dependence on the CSA also creates situations of "feast or famine" whereby firms increase capacity in response to a specific CSA contract and then must decrease capacity when the project is completed.

Notably, not all of the capacity developed has been Canadian because a number of subcontractors have been US-based firms. In many cases the necessary capacity was not available in Canada or not sufficiently evolved and thus presented too high a risk. In the view of some CSA representatives, firms involved in CASSIOPE (i.e., MDA and Magellan) had little choice but to work with US-based firms for some components because Canadian firms could not provide these parts.

With respect to the market potential of the Cascade technology, some CSA representatives questioned the long-term viability of the Cascade technology. As one interviewee commented, the Cascade technology "is both too late (obsolete) and too early." A few years ago there would have been a bigger market for the Cascade technology as a result of economic development in the North. However the increasing use of fibre optic cables has greatly decreased the market for Cascade and in many cases made it obsolete. Nevertheless there may still be a potential market for Cascade in the coming few years as a result of the melting polar ice cap which will see more ship traffic north of 60. Although geostationary satellites cover the North up to 82 degrees, there are challenges with bandwidth which does not support imagery. In terms of future satellites, it was reported that the Canadian-led Polar Communication and Weather (PCW) mission could (if it becomes a reality) provide the capability to transmit large volumes of data for large areas of Canada's North, due to its relatively large footprint, thus making Cascade obsolete.

4.2.2.3.2 Increased Capability of Models to Predict Space Weather

Although it is still premature to state conclusively that the data supplied via ePOP will contribute to an increased capability of models to predict space weather and its impact on radio transmission, there is good evidence this will likely occur.

According to representatives from the U of C, ePOP is already contributing knowledge about space weather that will be incorporated into weather prediction models to improve forecasts. U of C representatives explained that only recently has information on space weather been used for physics-based models on weather which provide more accurate forecasts. The sun is the main driver of weather and Canada's contribution internationally in solar physics is significant.

Although there is evidence this outcome will be achieved, it must be noted that scientific research is a long-term endeavour. Although the CASSIOPE satellite can be in orbit for a number of years (as long as its batteries will last), the collection of data for ePOP will depend on ongoing funding for data handling as well as ongoing interest within the research community in obtaining data from ePOP beyond the 18 months routine operating period stipulated in the MDA-University of Calgary contribution agreement.Footnote 10 Although ePOP is expected to make a positive contribution with respect to modelling space weather, it will be one of many sources of data over the coming few years. For example, as noted earlier, Swarm has been recently launched, and together with ePOP is contributing to knowledge of space weather.

4.2.2.3.3 Increased Security Resulting from Improved Prediction of Space Weather

There is some evidence that this outcome is beginning to be achieved using the early data emerging from ePOP.

Interviewees representing the U of C noted that ePOP data is being used to improve radio transmissions and the U of C is developing mitigation strategies when there is solar interference of transmissions. For example, space weather is severe over the poles and the ability to forecast solar flares is assisting airlines in determining better travel routes.

4.2.2.4 Achievement of Ultimate Outcomes
4.2.2.4.1 Expansion of Canada's Space Industry

Based on findings from interviews, CASSIOPE has had a positive, albeit moderate, impact on the space industry in Canada. CASSIOPE has allowed both the prime contractor and subcontractors to maintain existing capacity and, in some cases, expand product lines. It is, however, impossible to say whether the industry has expanded (or will in the future) as a result of CASSIOPE. As noted in previous sections, there is a limited market for Canadian space technology outside of the CSA with many companies such as Magellan almost entirely dependent on the CSA for business.

In addition to maintaining existing staff and capacity, CASSIOPE allowed both MDA and Magellan to develop new products and business lines. Although as noted previously the hardware designs are now somewhat obsolete, some of the skills and knowledge can be put to use in other missions. In fact the RCM project is using a smallsat bus developed by Magellan and Magellan was able to use some of what it learned on CASSIOPE in RCM (i.e., a complete re-engineering was not required). CSA representatives reported that Magellan has successfully leveraged the experience and expertise gained on CASSIOPE for other missions such as RCM. Many of the smallsat bus design elements and developmental processes used in CASSIOPE have been re-used in RCM and many of the relationships with subcontractors have been maintained by Magellan for RCM.

With respect to MDA's Cascade technology, as has been noted previously, if the market potential exists it will likely take an additional three to five years for MDA to develop that market.

4.2.2.4.2 Strengthening of Canada's Space Industry through Development of Capacity for Future Missions

Based on interview findings, there is evidence that CASSIOPE has had a positive impact on the space sector in Canada and has directly benefited the firms involved. Many of the relationships among firms developed through CASSIOPE have been maintained for the RCM project, for example. There is, however, no evidence that CASSIOPE has helped Canadian firms strengthen their position with respect to the international market for smallsats.

There is evidence that subcontractor and partner relationships have been developed among the firms involved in CASSIOPE. Magellan is reported to have developed strong relationships with its subcontractors. These relationships will have a positive impact on efficiency in working on future projects.

There is a sense among interviewees that many of these firms are very dependent on the CSA for their survival. These interviewees explained that beyond the CSA there normally are few clients nationally or internationally. The space industry tends not to be open internationally with most countries with an existing space sector providing their national firms with preferential treatment. Representatives from Magellan noted that they have not generated sales in the US despite tremendous efforts because the market in the US and elsewhere is basically closed. There is general agreement that countries tend to be protective of their domestic space industry making international markets for Canadian firms difficult to penetrate. The Emerson Report states that space and ground segment exports are flat and all of the increases in revenue for the sector are coming from applications and services.Footnote 11

Interviewees representing the CSA also noted that the space industry in Canada is quite small with only a few companies, many of which are not yet in a position to be international players. As noted by a few CSA representatives, MDA is an important player and the only credible prime contractor for large missions in Canada but is not large compared to the major international prime contractors in the space business.

4.2.2.4.3 Maintenance of Canadian Leadership in Space Environment Research

There is good evidence from key informant interviews that CASSIOPE has directly contributed to the maintenance of the leadership role of Canadian scientists in international research on space environment plasma dynamics and radio science.

There is agreement that the U of C has a particularly strong role internationally in the area of space environmental plasma dynamics and radio science. The ePOP team has established and maintained networks with international partners and the ePOP data will further strengthen these partnerships. CASSIOPE has already had an impact on international research through coordinating operations with ground facilities in other countries. Canada is seen as "punching above its weight class" with the U of C playing a strong role internationally. Several other Canadian research groups are also members of this scientific community, including from the University of Saskatchewan, the University of Alberta, the University of New Brunswick and Natural Resources Canada's (NRCan's) space weather forecasting centre, a part of the department's geomagnetic research laboratory. ePOP is the first time that eight instruments will be working together on a single platform to capture data on space weather and this is seen as contributing to Canada's position as a world leader in this area. The eight ePOP instruments were designed and built by a number of national and international partners representing academia, government and industry. These are summarized below:

Instrument Principal Investigator Institute and Industrial Partners
CER US Naval Research Lab
FAI University of Calgary, Burley Scientific, Routes AstroEngineering (now part of COMDEV)
GAP University of New Brunswick, Bristol Aerospace
IRM University of Calgary
MGF University of Calgary, Magnametrics, Bennest Enterprises, Narod Geophysics
NMS JAXA, MEISEI
RRI CRC, University of Calgary, COM DEV Ltd.
SEI University of Calgary

It was noted by a CSA representative that the delays in launching CASSIOPE have actually been beneficial to ePOP because the ESA launched Swarm in November 2013. Swarm is similar to ePOP and this will allow for a comparison of data sets, thereby increasing their precision and the value of the measurements obtained. A key investigator for both ePOP and Swarm is Dr. David Knudsen of U of C.

4.2.2.5 Appropriateness of Design of Management Structures

Based on evidence from the document review, a clearly defined governance structure was established for the CASSIOPE project. There are monitoring and reporting processes in place for the contribution agreements with both MDA and the U of C. However, both this current evaluation and the mid-term review completed in 2007 found that inadequate attention is being paid to the repayment of the contribution on the part of MDA.

Evidence from the key informant interviews suggests that the use of contribution agreements for CASSIOPE provided for the cost sharing of the three elements of CASSIOPE, none of which was likely to have been undertaken independently at the time. Although the use of contribution agreements for CASSIOPE represented a departure from the way the CSA typically implements missions, the approach is generally seen as having tangible benefits for both the CSA and the prime and its subcontractors.

The recipients of the two contributions prepared detailed plans for the management of the activities for which they are responsible. Performance monitoring of the CSA contribution to MDA and the U of C is a key element to ensure that orderly progress is being made according to the contributions' objectives. The contribution agreements make provision for adequate program management information:

  • Periodic program reviews with the University of Calgary.
  • Periodic program reviews with MDA/Cascade Data Services.
  • Periodic Integrated Program Reviews with all key stakeholders (i.e., University of Calgary, Cascade Data Services, MDA, EMS, COM DEV Ltd., CSA).
  • Monthly progress reports.

Representatives from the U of C feel that the CSA should use a structure that is more flexible and adaptive. In terms of management style, the U of C worked closely with both MDA and the CSA, and representatives feel that the MDA approach is more effective. The CSA management style is seen as more regimented and process orientated rather than results oriented.

Overall, most CSA and MDA representatives feel that the use of a contribution agreement approach was a good option. A few representatives from the CSA and MDA used words like "creative" and "innovative" in describing how the three components of CASSIOPE were combined via three contribution agreements. One MDA representative commented that the CASSIOPE project was an unusual way to share costs among the three objectives (ePOP, Cascade and the small satellite bus) but without this approach none of the three sub-projects would have been implemented. The combining of the Cascade component, ePOP and the smallsat bus represented a collaborative effort between the federal government, business and academia. The U of C participants believe they benefited by being exposed to the project management regime of MDA.

The CSA is accustomed to undertaking projects through contracts whereby the agency has much more influence over how the project is managed and implemented. Governance was simplified through the use of the contribution agreement mechanism; however, CASSIOPE was managed like a project (rather than a pure contribution) because of its high level of risk. Risk management on the part of the CSA of the MDA contribution agreement included site visits, monthly reporting, and input from outside experts to validate company reports.

In general terms, the decision to use a contribution agreement versus a contract depends on who benefits. A contribution agreement is used when the federal government gains no direct benefit – in the case of CASSIOPE, the ePOP component benefits the scientific community and the Cascade component benefits the industry. The CSA does not own any of the intellectual property (IP) rights to CASSIOPE satellite components.

The contribution agreement approach is seen to have had the following key benefits:

  • Improved working relationships. The prime contractor (i.e., MDA) was more engaged and there was more collaboration and a better working relationship between the CSA and the recipient of the contribution.
  • Less risk management on the part of the CSA. The CSA role under a contribution agreement is limited to oversight and this meant less risk management was required by the CSA relative to contractual agreements.
  • Decreased reporting requirements for prime contractor. A smaller reporting burden and more flexibility on the part of MDA in dealing with its subcontractors.
  • Funding was firmly set. All firms involved understood that there was no additional funding available regardless of challenges or issues encountered. This meant that everyone involved made the effort to resolve issues collaboratively.

The key challenge related to the use of a contribution agreement instead of a contractual agreement identified by CSA representatives is that the CSA has less direct influence.

An important aspect of the contribution agreement is the repayment to the Government of Canada of the contribution amount which is expected to flow from revenues from the commercialization of the Cascade technology. Under the terms of the contribution agreement, MDA is expected to repay 5% of the revenues from Cascade. The repayment schedule in the contribution agreement has not been updated yet despite the changes in launch dates. However, agreement was reached with MDA to extend the repayment end date to 2033 (instead of 2020). The original business plan produced by MDA is seen by some CSA representatives as having been overly optimistic and the current business plan is seen as missing major components such as an analysis of the marketplace for micro/nano satellites. There is little evidence from interviews or in the documentation reviewed of consistent efforts on the part of the CSA to monitor the repayment of the contribution by MDA. Findings from the 2007 mid-term review found evidence of the same gaps in managing the repayment.

Although the Cascade contribution agreement between the CSA and MDA indicates that contribution payments can be reduced if certain milestones related to attracting investor interest or investments are not met at the Launch Readiness Review, this "lever" was not used by the CSA. The use of this clause by the CSA would likely have stopped the project entirely and resulted in MDA not pursuing the Cascade project. In addition, ePOP would not have gone ahead and the CASSIOPE satellite would not have been launched.

4.2.2.6 Unintended Outcomes

The evaluation found limited evidence of unintended outcomes resulting from CASSIOPE. Some CSA interviewees indicated that locating the satellite ground station and operations at the University of Calgary served to broaden the capabilities of the university but worked contrary to the objective of CASSIOPE which is to develop industrial capacity. These interviewees feel that the ground station operations should have been managed by industrial partners in order to reflect the CSA's objective to develop or increase industrial capacity.

4.2.3 Demonstration of Efficiency and Economy

The TB Directive on the Evaluation Function defines the demonstration of efficiency and economy as the "assessment of resource utilization in relation to the production of outputs and progress towards expected outcomes." In general, the analysis of efficiency requires assessing relationships between inputs and outputs and/or outcomes, and the assessment of economy concerns the extent to which best use is made of resource inputs to achieve intended outcomes.

In practical terms, the key efficiency and economy questions of relevance to the CASSIOPE project were identified in the evaluation methodology design report as the following:

  • Was the budget for CASSIOPE appropriate/economical given the intended outputs to be produced and the mission's outcomes?
  • Did the project proceed in an efficient fashion?
  • How did the actual expenditures on CASSIOPE compare to the original budget?

No documentary evidence was provided by the CSA on the efficiency and economy issues, i.e., no previous assessments of the above sorts of questions were provided to the evaluation team. These issues also were not examined by the mid-term review.

Going back to the origins of CASSIOPE, the CSA and Industry Canada entered into negotiations with MDA, the prime contractor, to establish the two contribution agreements. The possibility of obtaining competitive proposals from other companies was not considered, as MDA was (and still is) the only prime contractor in Canada capable of undertaking such a complex mission involving the design, development, integration and launch of a small satellite with two payloads. In addition, the Cascade payload was an MDA-initiated product proposal, rather than originating with the CSA or Industry Canada.

The overall view of key informants is that the total budget for CASSIOPE was insufficient although no one could estimate the amount required. As described earlier, the total funding by the CSA and Industry Canada was $123.6 million (although additional funding was provided by NSERC and the CSA outside of the contribution agreement) and covered only hardware costs and operation (i.e., for the instruments and the operations centre), and not the costs of data analysis that would be incurred by U of C post-launch. It was noted that this was the typical approach of the CSA in the '90s and into the early 2000s. However, it is no longer the approach as the agency now takes a full life-cycle approach to designing and costing satellite missions.

Similarly, some feedback was received from industry that the costs allocated to the design and manufacture of the smallsat bus were also insufficient; in fact, the original cost estimate prepared by Magellan turned out to be lower than actual expenditures (according to Magellan, it had little time to prepare a price quotation to MDA at the proposal stage); consequently the company has had to invest its own funds to make up the difference and to maintain its contract with the prime contractor. The CSA receives a detailed summary of eligible costs from MDA on a regular basis. These cost summaries allow the CSA to track whether MDA is spending more than anticipated.

As discussed earlier in the results section, key informants within the CSA confirmed that all of the CASSIOPE project activities have been carried out and most of the outputs have been produced according to the requirements as laid out in the two contribution agreements.

The CSA did not modify the amount of funding provided to MDA over the life of the contribution agreement, and only added $1M to the U of C for the ePOP component. However, the CSA did provide some additional contract funding outside of the contribution agreement to U of C for data validation and analysis activities. In the view of CSA key informants (and the evaluation team) this funding was entirely appropriate, as it served to help maintain the U of C's research capability in the face of ongoing postponements to the CASSIOPE launch date. Without this additional funding, U of C might not have been able to keep the research team intact, which would have had significant negative repercussions for CASSIOPE's scientific mission. In fact, this contract funding helped the U of C be better prepared to commence its data analysis activities once CASSIOPE became finally operational in late 2013.

Perhaps an obvious finding pertaining to operational efficiency is that CASSIOPE's outputs were produced several years later than planned, and thus the achievement of most of the outcomes is also being delayed. As noted, there has however been a positive effect on ePOP outcomes from the delay in that ePOP's data can be analyzed together with data from the European satellite Swarm in real time.

With respect to economy, although it is premature to assume that the $63 million contribution to MDA will not be paid back in full, there is a risk that MDA will fail to secure sufficient customers to generate enough revenue to pay back the contribution. If MDA does not repay any of the contribution then the CSA will have spent $63 million to achieve the non-commercial outcomes resulting from the Cascade component of the CASSIOPE project, i.e., those outcomes which do not link to commercializing the Cascade technology. These non-commercial outcomes include maintaining and building capacity in the space technology industry and developing a design for a smallsat bus.

5. Conclusions and Recommendations

5.1 Relevance/Need

Missions such as CASSIOPE serve important commercial, societal and scientific needs of Canadians. The CASSIOPE contribution program contributes to Canada's societal and scientific needs by supporting the design and manufacture of world-class innovative technologies, products and services, and by providing training opportunities to young scientists and engineers. The objectives of the CASSIOPE project – design and manufacture of world-class technologies, training for young scientists and engineers, and promoting international partnerships – are well aligned with the federal government priorities as reflected in the recently released Space Policy Framework and the 2007 S&T Strategy.

The Aerospace Review Report ("Emerson Report") published in November 2012 provides a solid rationale for federal investment in the development of new satellites. The report explains that satellites are becoming more essential to modern economies and national security. Canada – with its vast geography, dispersed population, isolated communities, long coastlines, rich endowment of natural resources, and northern location – has a particular need for space assets and applications. The rationale for federal government support of projects such as CASSIOPE is echoed in the Space Policy Framework.

The influence of space weather is of particular concern for Canada, particularly because of the location of the North Magnetic Pole which lies just beyond the Canadian Arctic. This underscores the importance of research related to space weather such as ePOP.

5.2 Capacity

CASSIOPE has had a positive impact on the capacity of the space industry in Canada. CASSIOPE contributed directly to maintaining the existing capacity of the prime and subcontractors involved in the project and may have contributed to an expansion of their product lines. The development and maintenance of industrial capacity was an important objective for the CASSIOPE mission. Although it is intuitively obvious that CASSIOPE has had a positive impact on industrial capacity and there is some evidence to support this, quantitative performance data is not regularly collected.

Recommendation #1: The CSA should make efforts to track the impacts of its projects on industrial capacity. The performance measures should be relatively easy for industry partners to collect data and report on.

Internationally, the space industry is not open with most countries protecting their domestic industries and so Canadian firms are not easily able to develop an international market for their products. Domestically, Canadian firms are highly dependent on the CSA and other federal departments such as DND in order to maintain capacity. However the evaluation found evidence of frequent situations of "feast and famine." This makes it difficult for firms to maintain their capacity once it has been developed.

Recommendation #2: The CSA should explore ways to better ensure the Canadian space industry has the technological capacity to undertake space projects sponsored by the agency. This would align with the Space Policy Framework and the Aerospace Review, providing firms in the Canadian space sector and academia with direction on anticipated work and facilitating planning and capacity development, allowing the industry and academia to better respond to the needs of the CSA.

5.3 Technology and Business Risks

With the successful demonstration of the Cascade technology in February 2014, the technology risks have been fully retired.

With respect to the business risks, according to some, the technology has become somewhat obsolete over the past 10 years with the emergence of fibre optic technology, putting in doubt whether CASSIOPE can ever be competitive. However, others argue that the Cascade technology was never intended to compete with fibre optic technology. It was intended to efficiently transfer very large files from remote locations, ships and oil and gas rigs which are not served by fibre optics.

5.4 Scientific Research

The ePOP component of CASSIOPE is functioning as intended, and early data has been received enthusiastically by researchers. The ePOP data is seen as complementary to data being generated by the Swarm trio of satellites launched by the European Space Agency shortly after CASSIOPE in November 2013. There is optimism that ePOP data will make an important contribution over the coming few years (as long as CASSIOPE is in orbit) to the modelling of space weather and thus the security of radio transmissions, pipelines, electrical transmissions. The generation of ePOP data and the collaboration of the ePOP and Swarm research teams (led by the same key Canadian investigator) can be expected to result in international collaboration on the part of the U of C research team and a further cementing of Canada's leadership role in this area of research.

5.5 Design

Although the governance structure for CASSIOPE has worked well, there is evidence that the CSA did not exercise sufficient oversight of the repayment aspect of the MDA contribution agreement. The CASSIOPE project is the first project where the CSA has used a repayable contribution.

Recommendation #3: If the CSA intends to use repayable contributions in future space projects, it should ensure that it has the internal capacity to both assess the merits of the business case presented by the recipient and to manage the repayment process.

Although the evaluation found evidence of some challenges with the contribution agreement approach used for CASSIOPE, overall the use of contribution agreements was appropriate for CASSIOPE. The contribution agreement approach facilitated the implementation of three projects which may not have otherwise been implemented and represents a partnership between industry, academia and the federal government. The contribution agreement approach allowed for the sharing of risks across all partners and benefited contractors through decreased reporting requirements.

5.6 Economy and Efficiency

Total funding for CASSIOPE was $123.6 million with $12 million going towards ePOP and $111.6 million going towards the Cascade and the smallsat bus components. With the exception of an additional $1 million for ePOP, the value of the contribution agreements did not change over the 10-year period. It is very rare for space projects of the complexity of CASSIOPE to remain within budget as was the case for CASSIOPE. In the case of CASSIOPE, the use of a contribution agreement meant that industry had to absorb any cost overruns.

Management Response and Action Plan

  Responsibility
Organization / Function
Management response Details of action plan Schedule
Recommendation # 1
The CSA should make efforts to track the impacts of its projects on industrial capacity. The performance measures should be relatively easy for industry partners to collect data and report on.
  • Programs and Integrated Planning Directorate/ Director General
  • Policy/Director General
Agree A tool will be developed in line with the Space Policy Framework to measure the socio-economic benefits of CSA activities with industry, academia and other federal departments. December 2014
Recommendation # 2
The CSA should explore ways to better ensure the Canadian space industry has the technological capacity to undertake space projects sponsored by the agency. This would align with the Space Policy Framework and the Aerospace Review, providing firms and in the Canadian space sector and academia with direction on anticipated work and facilitating planning and capacity development, allowing the industry and academia to better respond to the needs of the CSA.
  • Programs and Integrated Planning Directorate/ Director General
  • Policy/ Director General
Agree
  • Complete the CSA Space Strategy and make it publically available for industry and academia to plan in the long term.
  • The CSA will lead an Annual Space Conference with all Canadian key stakeholders. The first one is on Feb 25, 2014, at the CSA.
  • March 2015
  • Ongoing
Recommendation # 3
If the CSA intends to use repayable contributions in future space projects, it should ensure that it has the internal capacity to both assess the merits of the business case presented by the recipient and to manage the repayment process. Space Science & Technology Directorate/ Director General Agree
  • Review of the Grants & Contributions Centre of Expertise's (G&C COE) roles and responsibilities as well as its capabilities and structure within the CSA.
  • The G&C COE will establish partnerships and work closely with ITO (IC) in order to harmonize the Government of Canada approach regarding CASSIOPE's repayment process as well as to benefit from ITO expertise in that matter.
  • A framework to assess the merits of the business case as well as a repayment framework will be developed by the G&C COE in line with the Transfer Payment Directives.
  • June 2014
  • Jan 2016
  • March 2016

Appendices

Appendix A: Logic Model and Narrative

The logic model for the CASSIOPE Contribution Agreement for the CSA is presented in Figure A-1. The corresponding narrative is presented below.

Activities

Cascade Space and Ground Segment Design and Development

Activities conducted by CDS/MDA related to the design, manufacture and testing of the:

  • Cascade Ka-Band payload including the data storage unit;
  • Transportable ground terminal for demonstration of the Cascade service; and
  • Experimental service control centre for demonstration of the Cascade service.
CASSIOPE/Smallsat Bus Detailed Design and Manufacturing

Activities conducted by CDS/MDA and related to the development of the:

  • Small satellite bus including the mechanical structure, solar panels and an altitude control system, suitable for Cascade and ePOP payloads.
ePOP Payload Design and Development

Activities conducted by the University of Calgary and related to the:

  • Design, manufacture and testing of six science instruments (e.g., Imaging Ion-Mass Spectrometer, Suprathermal Electron Imager, Fast Auroral Imager, Radio Receiver Instrument, GPS Altitude and Position Instrument, and a 3-axis Fuxgate Magnetometer);
  • Design, manufacture and testing of a Data Handling Unit; and
  • Integration of two instruments provided by Japan (e.g., Neutral Mass Spectrometer) and by the US Naval Research Laboratories (e.g., Cohert Electromagnetic Radio Tomography).
CASSIOPE Mission – Design, Integration, Ground Segment and Operations

Activities conducted by CDS/MDA and related to the:

  • Design engineering for the joint ePOP-Cascade Technology Demonstration Mission;
  • Assembly, integration and testing (AIT) of the Cascade and ePOP payloads on the Smallsat bus;
  • Procurement of a launcher for the satellite and in-orbit commissioning;
  • Satellite telemetry, tracking and control; and
  • Cascade and ePOP payload operations.

Outputs

Cascade Payload
  • Delivery of the components and sub-systems (e.g., space-qualified modulators and demodulators, disk-drive, Ka-band RF chains, etc.) required for building the Cascade payload, by the companies participating in CASSIOPE R&D activities.
  • Delivery of the components and sub-systems required for building the Cascade portable ground terminals and Experimental Service Control Centre.
  • On-orbit operations of the Cascade payload and demonstration of a digital satellite courier service as per Requirements.
Small Satellite Bus
  • Design and manufacturing of a Smallsat bus.
  • On-orbit operations of the CASSIOPE spacecraft during one year and demonstration of the small satellite bus capabilities and associated performance data.
ePOP
  • Delivery of the eight scientific instruments and the Data Handling Unit required for the building the ePOP payload, by the companies, universities and foreign organizations participating in CASSIOPE R&D activities.
  • On-orbit operations of the ePOP payload with its eight scientific instruments and production of the scientific data to the University of Calgary.

Immediate Outcomes

Cascade Payload
  • Reduction or retirement of the technology risks associated with the Cascade digital satellite courier service through on-orbit demonstration.
  • Reduction or retirement of the market risks associated with the Cascade digital satellite courier service through the Early Entry Strategy.
  • Establishment of the viability of the Cascade service technology and market with prospective users from the target niche market, i.e., the seismic vessels involved in oil and gas exploration.
  • Attraction of the outside investments needed for the construction of the future four satellites of the pre-production phase.
Small Satellite Bus
  • Suitability of the first Canadian small satellite bus for the construction of future Cascade satellites through the successful one-year operation of the CASSIOPE Smallsat bus.
Cascade Payload and Small Satellite Bus
  • Reduction of recurring manufacturing costs and timelines for the future four satellites of the pre-production phase through the establishment of a set of qualified Canadian suppliers.
ePOP
  • Acquisition of new fundamental scientific knowledge on plasma dynamics in Earth's ionosphere through analysis on ePOP data; and enhanced understanding of the effect of plasma dynamics on radio transmissions, transport of gases and Earth's climate through analysis on the ePOP data.

Intermediary Outcomes

Cascade Payload
  • Creation of a new export-oriented global satellite-based courier service built and operated in Canada.
  • Maintenance of the core manufacturing capabilities of the Canadian satellite communication industry.
Small Satellite Bus
  • Creation of a competitive bus supplier in Canada capable to designing and manufacturing small satellite buses for future Canadian and/or international missions.
ePOP
  • Increased capabilities of models to predict space weather, and its impact on and distortion of radio transmissions.
  • Increased security of radio transmissions, pipelines and electrical transmission systems through better prediction of space weather models.

Ultimate Outcomes

  • Expansion of Canada's space industry through the development of advanced technologies and equipment created for CASSIOPE, as well as the establishment of a new advanced global satellite courier service and a bus manufacturing capability.
  • Strengthening of the Canadian space industry with the development of a competitive prime contractor/ systems integrator capacity and a team of effective suppliers to serve future commercial and scientific missions.
  • Maintenance of the leadership role played by Canadian scientists in leading-edge international researches on space environment plasma dynamics and radio science.

Final Outcome

  • Contribution to the Government of Canada Whole-of-Government Framework outcomes of strong economic growth and an innovative and knowledge-based economy:
    • by supporting the design and manufacture of world-class innovative technologies, products and/or services,
    • by providing training opportunities to many young scientists and engineers,
    • by promoting Canadian industry abroad through exposure to international partnerships.

Figure A-1: Logic Model for CASSIOPE Contribution Program

Appendix B: Documents Reviewed

CASSIOPE Project Team Documentation

  • 51-5187 - CASSIOPE Monthly Report Sep_09
  • 51-5187 - CASSIOPE Monthly Report Sept_2011 - Issue-94-0
  • 51-5187 CASSIOPE Monthly Report June 2013 Issue -115 0
  • 51-5187 CASSIOPE Monthly Report Sept_2012 Issue-106 0
  • Annual-update-2013
  • Biannual-subcontractor-report-2012-12-31
  • CASSIOPE - MEMO to MIN - DM SIGNED
  • CASSIOPE amend_6 contribution _agreement(signed)
  • CASSIOPE Amendment 5 Contribution Agreement 2009-07-31
  • CASSIOPE Contribution Agreement Final signed
  • CASSIOPE Memo to the Minister 23 Feb 2012_Final SIGNED
  • CASSIOPE Mission contribution agreement amendment 1
  • CASSIOPE Mission contribution agreement amendment 2
  • CASSIOPE Mission contribution agreement amendment 3
  • CASSIOPE Mission contribution agreement amendment 4
  • CASSIOPE Mission contribution agreement amendment 7
  • CASSIOPE SC Processing Schedule
  • CASSIOPE mission(20130605).ppt
  • CASSIOPE_Re_Awake_Review_Agenda_2012_07_26
  • CASSIOPE_Re_Awake_Review_Presentation
  • Definition of substandard satellite operation - 2006-05
  • ePOP Amendment number 3 2007-12-15
  • ePOP Contribution Agreement 2004-02-06
  • ePOP amendment number 5 Contribution Agreement 2009-08-01
  • ePOP amendment number 1 2005-09-01
  • ePOP amendment number 2 2006-11-01
  • ePOP amendment number 4 2008-09-01
  • ePOP_Amendment_number_six_(signed)
  • Ground Segment Ops Schedule 04
  • MDA Presentation QPR#25_07_Sep_2012_pptx
  • Min Prentice RE CASSIOPE Extension
  • Monthly Progress Report-June-2013
  • Monthly Progress Report-Sep-2012
  • Monthly Progress Report-September-2011
  • SpaceX_CASSIOPE Quarterly Review - 31Aug2012
  • TB Sub 21-10-03 finale_angl_fr.avec changements_14h30
  • TB_sub_CASSIOPE_final_draft_(clean)(20130814)

CSA Administrative Documentation

External Documentation

Appendix C: Interview Guides

Evaluation of the CASSIOPE Project – Interview Guide – Interviews within the Canadian Space Agency

A. Background

Thank you for agreeing to be interviewed as part of the Evaluation of the CASSIOPE project. The Canadian Space Agency (CSA) has engaged a team of outside evaluation specialists to conduct this independent evaluation.

The CASSIOPE project comprises three working elements that use the first multi-purpose small satellite platform from the Canadian Small Satellite Bus Program. This generic, low-cost platform carries two payloads: the Enhanced-Polar Outflow Probe (ePOP) scientific payload and Cascade. The ePOP payload consists of eight high-resolution instruments used to probe the characteristics of near-Earth space. Cascade is a high data rate, high capacity store and forward technology payload.

MacDonald, Dettwiler and Associates (MDA) and its subsidiary Cascade Data Services (CDS) were involved in the technology demonstration aspects of CASSIOPE. University of Calgary developed the ePOP elements.

The objectives of the CASSIOPE project are to:

  • Demonstration of the viability of a global digital satellite courier service, capable of picking up and delivering extremely large data packages (50 to 500 gigabytes per day) with delivery guaranteed within a day anywhere in the world.
  • Development of a world-class small satellite bus.
  • Preservation of critical space knowledge, expertise and industrial capacity in Canada.
  • Acquiring fundamental new knowledge relating to the dynamics of the Earth's space plasma environment, effects on radio wave propagation, and relations between the Sun and the Earth's climate.
  • Further development of a prime contractor and systems integrator capability in Canadian industry.

The total estimated funding for the CASSIOPE project is $81 million.

The objectives of the evaluation study are to assess the continued relevance of the objectives of the CASSIOPE project and its performance in achieving those objectives. The Canadian Space Agency must complete the evaluation of the CASSIOPE project by the end of March 2014 in order to respond to Treasury Board requirements.

The evaluation involves a number of data collection activities, including interviews within the CSA and with several partners, members of the science team and potential users of data that will be generated by CASSIOPE.

Your views will be kept confidential by the evaluation team, and only aggregated results will be included in the evaluation report. Once approved, the final evaluation report will be made public by the CSA in accordance with Treasury Board policy.

Your interview is expected to last up to one hour.

B. Questions

Please review the following questions in advance of your interview. If you have no opinion on a particular question, feel free to skip it.

  1. Alignment with government priorities. How do the overall objectives of the CASSIOPE project (stated above) currently fit with CSA and broader federal government priorities such as the Federal S&T Strategy, job creation, etc.? Are there any documents you can point to that would help demonstrate this alignment?
  2. Government funding of space R&D. Is the level of funding a barrier to the conduct of space-related R&D? Do you believe it is appropriate for the federal government to stimulate R&D in the space industry by directly funding R&D?
  3. Implementation. To what extent has the CASSIOPE project been implemented as initially planned? What, if any, challenges have been encountered? How were these challenges addressed?
  4. Outputs. Has CASSIOPE produced – or likely to produce – the following outputs:
    1. Delivery of components and sub-systems (e.g., space-qualified modulators and demodulators, disk-drive, Ka-band RF chains, etc.) required for building the Cascade payload.
    2. Delivery of components and sub-systems required for building the Cascade portable ground terminals and Experimental Service Control Centre.
    3. Design and manufacture of a SmallSat bus.
    4. Delivery of the eight scientific instruments and data handling unit required for building the ePOP payload.
    5. On-Orbit orbit operations of the Cascade payload and demonstration of a digital satellite courier service.
    6. On-orbit operation of the CASSIOPE spacecraft, including demonstration of the small satellite bus capabilities required for effectively carrying both Cascade and ePOP payloads.
    7. On-orbit operations of the ePOP payload with its eight scientific instruments and the data handling unit required for building the ePOP payload?
  5. Immediate outcomes. Has CASSIOPE achieved – or likely to achieve – the following short-term outcomes:
    1. Reduce the technology risks associated with Cascade digital satellite courier service?
    2. Improve the suitability of the first Canadian small satellite bus for the construction of future Cascade satellites? In your opinion, how likely is it that the small satellite bus will be suitable for construction of future Cascade satellites?
    3. Contribute to the acquisition of scientific knowledge through ePOP data (e.g., knowledge of plasma dynamics in the Earth's ionosphere and increased understanding of the effects of plasma dynamics on radio transmission, transport of gases and the Earth's climate?
  6. Intermediate outcomes. Is CASSIOPE likely to achieve the following intermediate outcomes:
    1. Contribute to maintaining the core manufacturing capabilities of the Canadian satellite communication industry (e.g., impacts on the companies participating in CASSIOPE). To what extent have income and employment of the satellite communications companies participating in CASSIOPE been maintained at the level existing in 2003?
    2. The data supplied via ePOP contributes to an increased capability of models to predict space weather and its impact on radio transmissions.
    3. Data supplied via ePOP contributes to improved prediction of space weather.
    4. Data supplied via ePOP contributes to increased security of radio transmissions, pipelines and electrical transmission system?
  7. Ultimate outcomes. Finally, is CASSIOPE likely to achieve its longer-term outcomes:
    1. Contribute to an expansion of Canada's space industry through development of advanced technologies and equipment? For example, has there been a growth in revenues of the companies participating in CASSIOPE due to the project?
    2. Strengthened Canada's space industry as a result of the development of the prime contractor and its suppliers?
    3. ePOP contributes to the maintenance of the leadership role of Canadian scientists in research on space environment plasma dynamics and radio science? Might CASSIOPE also have a positive impact on international research?
  8. Design. Is the CASSIOPE project design appropriate for ensuring the mission is successful? Is the governance structure working?
  9. Unintended impacts. To your knowledge, have there been any unintended impacts from the CASSIOPE project (either positive or negative)?
  10. Economy and efficiency:
    1. Could CASSIOPE's outputs have been produced at a lower cost? How appropriate were the amounts of financial and human resources allocated to CASSIOPE?
    2. Could the project's activities been carried out in a more efficient manner?
    3. Might there have been entirely different ways of achieving the intended outcomes at a lower cost?
  11. Other comments. Finally, do you have any other comments regarding CASSIOPE that have not been covered above?

Evaluation of the CASSIOPE Project – Interview Guide – Interviews within Industry Canada Representatives

A. Background

Thank you for agreeing to be interviewed as part of the Evaluation of the CASSIOPE project. The Canadian Space Agency (CSA) has engaged a team of outside evaluation specialists to conduct this independent evaluation.

The CASSIOPE project comprises three working elements that use the first multi-purpose small satellite platform from the Canadian Small Satellite Bus Program. This generic, low-cost platform carries two payloads: the Enhanced-Polar Outflow Probe (ePOP) scientific payload and Cascade. The ePOP payload consists of eight high-resolution instruments used to probe the characteristics of near-Earth space. Cascade is a high data rate, high capacity store and forward technology payload.

MacDonald, Dettwiler and Associates (MDA) and its subsidiary Cascade Data Services (CDS) were involved in the technology demonstration aspects of CASSIOPE. University of Calgary developed the ePOP elements.

The objectives of the CASSIOPE project are to:

  • Demonstration of the viability of a global digital satellite courier service, capable of picking up and delivering extremely large data packages (50 to 500 gigabytes per day) with delivery guaranteed within a day anywhere in the world.
  • Development of a world-class small satellite bus.
  • Preservation of critical space knowledge, expertise and industrial capacity in Canada.
  • Acquiring fundamental new knowledge relating to the dynamics of the Earth's space plasma environment, effects on radio wave propagation, and relations between the Sun and the Earth's climate.
  • Further development of a prime contractor and systems integrator capability in Canadian industry.

The total estimated funding for the CASSIOPE project is $81 million.

The objectives of the evaluation study are to assess the continued relevance of the objectives of the CASSIOPE project and its performance in achieving those objectives. The Canadian Space Agency must complete the evaluation of the CASSIOPE project by the end of March 2014 in order to respond to Treasury Board requirements.

The evaluation involves a number of data collection activities, including interviews within the CSA and with several partners, members of the science team and potential users of data that will be generated by CASSIOPE.

Your views will be kept confidential by the evaluation team, and only aggregated results will be included in the evaluation report. Once approved, the final evaluation report will be made public by the CSA in accordance with Treasury Board policy.

Your interview is expected to last up to one hour.

B. Questions

Please review the following questions in advance of your interview. If you have no opinion on a particular question, feel free to skip it.

  1. Government funding of space R&D. Is the level of funding a barrier to the conduct of space-related R&D? Do you believe it is appropriate for the federal government to stimulate R&D in the space industry by directly funding R&D?
  2. Implementation. To what extent has the CASSIOPE project been implemented as initially planned? What, if any, challenges have been encountered? How were these challenges addressed?
  3. Intermediate outcomes. Is CASSIOPE likely to achieve the following intermediate outcomes:
    1. Contribute to maintaining the core manufacturing capabilities of the Canadian satellite communication industry (e.g., impacts on the companies participating in CASSIOPE).
  4. Ultimate outcomes. Finally, is CASSIOPE likely to achieve its longer-term outcomes:
    1. Contribute to an expansion of Canada's space industry through development of advanced technologies and equipment? For example, has there been a growth in revenues of the companies participating in CASSIOPE due to the project?
    2. Strengthened Canada's space industry as a result of the development of the prime contractor and its suppliers?
    3. ePOP contributes to the maintenance of the leadership role of Canadian scientists in research on space environment plasma dynamics and radio science? Might CASSIOPE also have a positive impact on international research?
  5. Design. Is the CASSIOPE project design appropriate for ensuring the mission is successful? Is the governance structure working?
  6. Unintended impacts. To your knowledge, have there been any unintended impacts from the CASSIOPE project (either positive or negative)?
  7. Economy and efficiency.
    1. Could CASSIOPE's outputs have been produced at a lower cost? How appropriate were the amounts of financial and human resources allocated to CASSIOPE?
    2. Could the project's activities been carried out in a more efficient manner?
    3. Might there have been entirely different ways of achieving the intended outcomes at a lower cost?
  8. Other comments. Finally, do you have any other comments regarding CASSIOPE that have not been covered above?

Evaluation of the CASSIOPE Project – Interview Guide – Interviews with Representatives from Magellan

A. Background

Thank you for agreeing to be interviewed as part of the Evaluation of the CASSIOPE project. The Canadian Space Agency (CSA) has engaged a team of outside evaluation specialists to conduct this independent evaluation.

The CASSIOPE project comprises three working elements that use the first multi-purpose small satellite platform from the Canadian Small Satellite Bus Program. This generic, low-cost platform carries two payloads: the Enhanced-Polar Outflow Probe (ePOP) scientific payload and Cascade. The ePOP payload consists of eight high-resolution instruments used to probe the characteristics of near-Earth space. Cascade is a high data rate, high capacity store and forward technology payload.

MacDonald, Dettwiler and Associates (MDA) and its subsidiary Cascade Data Services (CDS) were involved in the technology demonstration aspects of CASSIOPE. University of Calgary developed the ePOP elements.

The objectives of the CASSIOPE project are to:

  • Demonstration of the viability of a global digital satellite courier service, capable of picking up and delivering extremely large data packages (50 to 500 gigabytes per day) with delivery guaranteed within a day anywhere in the world.
  • Development of a world-class small satellite bus.
  • Preservation of critical space knowledge, expertise and industrial capacity in Canada.
  • Acquiring fundamental new knowledge relating to the dynamics of the Earth's space plasma environment, effects on radio wave propagation, and relations between the Sun and the Earth's climate.
  • Further development of a prime contractor and systems integrator capability in Canadian industry.

The total estimated funding for the CASSIOPE project is $81 million.

The objectives of the evaluation study are to assess the continued relevance of the objectives of the CASSIOPE project and its performance in achieving those objectives. The Canadian Space Agency must complete the evaluation of the CASSIOPE project by the end of March 2014 in order to respond to Treasury Board requirements.

The evaluation involves a number of data collection activities, including interviews within the CSA and with several partners, members of the science team and potential users of data that will be generated by CASSIOPE.

Your views will be kept confidential by the evaluation team, and only aggregated results will be included in the evaluation report. Once approved, the final evaluation report will be made public by the CSA in accordance with Treasury Board policy.

Your interview is expected to last up to one hour.

B. Questions

Please review the following questions in advance of your interview. If you have no opinion on a particular question, feel free to skip it.

  1. Government funding of space R&D. Is the level of funding a barrier to the conduct of space-related R&D? Do you believe it is appropriate for the federal government to stimulate R&D in the space industry by directly funding R&D?
  2. Outputs. Has CASSIOPE produced – or likely to produce – the following outputs:
    1. Design and manufacture of a SmallSat bus.
    2. On-orbit operation of the CASSIOPE spacecraft, including demonstration of the small satellite bus capabilities required for effectively carrying both Cascade and ePOP payloads.
  3. Immediate outcomes. Has CASSIOPE achieved – or likely to achieve – the following short-term outcomes:
    1. Improve the suitability of the first Canadian small satellite bus for the construction of future Cascade satellites? In your opinion, how likely is it that the small satellite bus will be suitable for construction of future Cascade satellites?
  4. Intermediate outcomes. Is CASSIOPE likely to achieve the following intermediate outcomes:
    1. Contribute to maintaining the core manufacturing capabilities of the Canadian satellite communication industry (e.g., impacts on the companies participating in CASSIOPE). To what extent have income and employment of the satellite communications companies participating in CASSIOPE been maintained at the level existing in 2003?
  5. Ultimate outcomes. Finally, is CASSIOPE likely to achieve its longer-term outcomes:
    1. Contribute to an expansion of Canada's space industry through development of advanced technologies and equipment? For example, has there been a growth in revenues of the companies participating in CASSIOPE due to the project?
  6. Ultimate impact on industry: Are you able to estimate the number of young scientists and engineers provided with training opportunities as a direct result of CASSIOPE? Are you able to estimate the number of partnerships resulting from CASSIOPE? To what extent has CASSIOPE facilitated (or is likely to facilitate) the promotion of the Canadian space industry internationally?
  7. Unintended impacts. To your knowledge, have there been any unintended impacts from the CASSIOPE project (either positive or negative)?
  8. Other comments. Finally, do you have any other comments regarding CASSIOPE that have not been covered above?

Evaluation of the CASSIOPE Project – Interview Guide – Interviews with Representatives from MDA/Cascade

A. Background

Thank you for agreeing to be interviewed as part of the Evaluation of the CASSIOPE project. The Canadian Space Agency (CSA) has engaged a team of outside evaluation specialists to conduct this independent evaluation.

The CASSIOPE project comprises three working elements that use the first multi-purpose small satellite platform from the Canadian Small Satellite Bus Program. This generic, low-cost platform carries two payloads: the Enhanced-Polar Outflow Probe (ePOP) scientific payload and Cascade. The ePOP payload consists of eight high-resolution instruments used to probe the characteristics of near-Earth space. Cascade is a high data rate, high capacity store and forward technology payload.

MacDonald, Dettwiler and Associates (MDA) and its subsidiary Cascade Data Services (CDS) were involved in the technology demonstration aspects of CASSIOPE. University of Calgary developed the ePOP elements.

The objectives of the CASSIOPE project are to:

  • Demonstration of the viability of a global digital satellite courier service, capable of picking up and delivering extremely large data packages (50 to 500 gigabytes per day) with delivery guaranteed within a day anywhere in the world.
  • Development of a world-class small satellite bus.
  • Preservation of critical space knowledge, expertise and industrial capacity in Canada.
  • Acquiring fundamental new knowledge relating to the dynamics of the Earth's space plasma environment, effects on radio wave propagation, and relations between the Sun and the Earth's climate.
  • Further development of a prime contractor and systems integrator capability in Canadian industry.

The total estimated funding for the CASSIOPE project is $81 million.

The objectives of the evaluation study are to assess the continued relevance of the objectives of the CASSIOPE project and its performance in achieving those objectives. The Canadian Space Agency must complete the evaluation of the CASSIOPE project by the end of March 2014 in order to respond to Treasury Board requirements.

The evaluation involves a number of data collection activities, including interviews within the CSA and with several partners, members of the science team and potential users of data that will be generated by CASSIOPE.

Your views will be kept confidential by the evaluation team, and only aggregated results will be included in the evaluation report. Once approved, the final evaluation report will be made public by the CSA in accordance with Treasury Board policy.

Your interview is expected to last up to one hour.

B. Questions

Please review the following questions in advance of your interview. If you have no opinion on a particular question, feel free to skip it.

  1. Government funding of space R&D. Is the level of funding a barrier to the conduct of space-related R&D? Do you believe it is appropriate for the federal government to stimulate R&D in the space industry by directly funding R&D?
  2. Implementation. To what extent has the CASSIOPE project been implemented as initially planned? What, if any, challenges have been encountered? How were these challenges addressed?
  3. Outputs. Has CASSIOPE produced – or likely to produce – the following outputs:
    1. Delivery of components and sub-systems (e.g., space-qualified modulators and demodulators, disk-drive, Ka-band RF chains, etc.) required for building the Cascade payload.
    2. Delivery of components and sub-systems required for building the Cascade portable ground terminals and Experimental Service Control Centre.
    3. Design and manufacture of a SmallSat bus.
    4. On-Orbit orbit operations of the Cascade payload and demonstration of a digital satellite courier service.
    5. On-orbit operation of the CASSIOPE spacecraft, including demonstration of the small satellite bus capabilities required for effectively carrying both Cascade and ePOP payloads.
  4. Immediate outcomes. Has CASSIOPE achieved – or likely to achieve – the following short-term outcomes:
    1. Reduce the technology risks associated with Cascade digital satellite courier service?
    2. Improve the suitability of the first Canadian small satellite bus for the construction of future Cascade satellites? In your opinion, how likely is it that the small satellite bus will be suitable for construction of future Cascade satellites?
  5. Intermediate outcomes. Is CASSIOPE likely to achieve the following intermediate outcomes:
    1. Contribute to maintaining the core manufacturing capabilities of the Canadian satellite communication industry (e.g., impacts on the companies participating in CASSIOPE). To what extent have income and employment of the satellite communications companies participating in CASSIOPE been maintained at the level existing in 2003?
  6. Ultimate outcomes. Finally, is CASSIOPE likely to achieve its longer-term outcomes:
    1. Contribute to an expansion of Canada's space industry through development of advanced technologies and equipment? For example, has there been a growth in revenues of the companies participating in CASSIOPE due to the project?
    2. Strengthened Canada's space industry as a result of the development of the prime contractor and its suppliers?
  7. Ultimate impact on industry: Are you able to estimate the number of young scientists and engineers provided with training opportunities as a direct result of CASSIOPE? Are you able to estimate the number of partnerships resulting from CASSIOPE? To what extent has CASSIOPE facilitated (or is likely to facilitate) the promotion of the Canadian space industry internationally?
  8. Design. Is the CASSIOPE project design appropriate for ensuring the mission is successful? Is the governance structure working?
  9. Unintended impacts. To your knowledge, have there been any unintended impacts from the CASSIOPE project (either positive or negative)?
  10. Economy and efficiency:
    1. Could CASSIOPE's outputs have been produced at a lower cost? How appropriate were the amounts of financial and human resources allocated to CASSIOPE?
    2. Could the project's activities been carried out in a more efficient manner?
    3. Might there have been entirely different ways of achieving the intended outcomes at a lower cost?
  11. Other comments. Finally, do you have any other comments regarding CASSIOPE that have not been covered above?

Evaluation of the CASSIOPE Project – Interview Guide – Interviews Representatives from the University of Calgary

A. Background

Thank you for agreeing to be interviewed as part of the Evaluation of the CASSIOPE project. The Canadian Space Agency (CSA) has engaged a team of outside evaluation specialists to conduct this independent evaluation.

The CASSIOPE project comprises three working elements that use the first multi-purpose small satellite platform from the Canadian Small Satellite Bus Program. This generic, low-cost platform carries two payloads: the Enhanced-Polar Outflow Probe (ePOP) scientific payload and Cascade. The ePOP payload consists of eight high-resolution instruments used to probe the characteristics of near-Earth space. Cascade is a high data rate, high capacity store and forward technology payload.

MacDonald, Dettwiler and Associates (MDA) and its subsidiary Cascade Data Services (CDS) were involved in the technology demonstration aspects of CASSIOPE. University of Calgary developed the ePOP elements.

The objectives of the CASSIOPE project are to:

  • Demonstration of the viability of a global digital satellite courier service, capable of picking up and delivering extremely large data packages (50 to 500 gigabytes per day) with delivery guaranteed within a day anywhere in the world.
  • Development of a world-class small satellite bus.
  • Preservation of critical space knowledge, expertise and industrial capacity in Canada.
  • Acquiring fundamental new knowledge relating to the dynamics of the Earth's space plasma environment, effects on radio wave propagation, and relations between the Sun and the Earth's climate.
  • Further development of a prime contractor and systems integrator capability in Canadian industry.

The total estimated funding for the CASSIOPE project is $81 million.

The objectives of the evaluation study are to assess the continued relevance of the objectives of the CASSIOPE project and its performance in achieving those objectives. The Canadian Space Agency must complete the evaluation of the CASSIOPE project by the end of March 2014 in order to respond to Treasury Board requirements.

The evaluation involves a number of data collection activities, including interviews within the CSA and with several partners, members of the science team and potential users of data that will be generated by CASSIOPE.

Your views will be kept confidential by the evaluation team, and only aggregated results will be included in the evaluation report. Once approved, the final evaluation report will be made public by the CSA in accordance with Treasury Board policy.

Your interview is expected to last up to one hour.

B. Questions

Please review the following questions in advance of your interview. If you have no opinion on a particular question, feel free to skip it.

  1. Government funding of space R&D. Is the level of funding a barrier to the conduct of space-related R&D? Do you believe it is appropriate for the federal government to stimulate R&D in the space industry by directly funding R&D?
  2. Implementation. To what extent has the CASSIOPE project been implemented as initially planned? What, if any, challenges have been encountered? How were these challenges addressed?
  3. Outputs. Has CASSIOPE produced – or likely to produce – the following outputs:
    1. Delivery of the eight scientific instruments and data handling unit required for building the ePOP payload.
    2. On-orbit operations of the ePOP payload with its eight scientific instruments and the data handling unit required for building the ePOP payload?
  4. Immediate outcomes. Has CASSIOPE achieved – or likely to achieve – the following short-term outcome:
    1. Contribute to the acquisition of scientific knowledge through ePOP data (e.g., knowledge of plasma dynamics in the Earth's ionosphere and increased understanding of the effects of plasma dynamics on radio transmission, transport of gases and the Earth's climate?
  5. Intermediate outcomes. Is CASSIOPE likely to achieve the following intermediate outcomes:
    1. The data supplied via ePOP contributes to an increased capability of models to predict space weather and its impact on radio transmissions.
    2. Data supplied via ePOP contributes to improved prediction of space weather.
    3. Data supplied via ePOP contributes to increased security of radio transmissions, pipelines and electrical transmission system?
  6. Ultimate outcomes. Finally, is CASSIOPE likely to achieve its longer-term outcome:
    1. ePOP contributes to the maintenance of the leadership role of Canadian scientists in research on space environment plasma dynamics and radio science? Might CASSIOPE also have a positive impact on international research?
  7. Design. Is the CASSIOPE project design appropriate for ensuring the mission is successful? Is the governance structure working?
  8. Unintended impacts. To your knowledge, have there been any unintended impacts from the CASSIOPE project (either positive or negative)?
  9. Economy and efficiency:
    1. Could CASSIOPE's outputs have been produced at a lower cost? How appropriate were the amounts of financial and human resources allocated to CASSIOPE?
    2. Could the project's activities been carried out in a more efficient manner?
    3. Might there have been entirely different ways of achieving the intended outcomes at a lower cost?
  10. Other comments. Finally, do you have any other comments regarding CASSIOPE that have not been covered above?

Appendix D: Evaluation Matrix

Evaluation Issues, Indicators, Data Sources/Methods

Relevance - Does the program remain consistent with and contribute to the federal government priorities and address actual needs?

Question Indicators Sources/Methods
1. Is there a continued need for the CSA to be involved in investing in a project such as CASSIOPE? 1.1 Assessment/demonstration of ongoing societal and scientific needs for the CASSIOPE project.
  • Document review (e.g., Treasury Board Submission and amendments).
2. Is the CASSIOPE project aligned with federal government priorities? 2.1 Extent to which CASSIOPE is aligned with the 2007 Federal S&T Strategy and other federal priorities, including job creation.
  • Document review (e.g., Federal S&T Strategy, DPR/RPP).
  • Key informant interviews:
    • CSA managers and staff.
2.2 Extent to which CASSIOPE's objectives are aligned with current CSA priorities.
  • Document review (e.g., RPP/DPR, current PAA).
  • Key informant interviews:
    • CSA managers and staff
3. Is the CASSIOPE project consistent with federal roles and responsibilities? 3.1 Extent to which it is appropriate for CASSIOPE to fall under federal government jurisdiction.
  • Document review (e.g., the CSA Act 1990, TB Submission).
3.2 Extent to which it is appropriate for the federal government to support the space technology industry.
  • Document review (e.g., TB Submission, Aerospace Review Report Volumes 1 & 2).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada)
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.

Performance - Has the program achieved its intended outcomes? Are the most appropriate, efficient and economic means being used to achieve outcomes?

Question Indicators Sources/Methods
4. To what extent have CASSIOPE activities been implemented as intended? 4.1 Extent to which CASSIOPE activities have been implemented as originally planned.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.
4.2 Challenges encountered in implementing CASSIOPE activities and implications for success of CASSIOPE.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.
4.3 Extent to which challenges were mitigated.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.
5. To what extent has CASSIOPE produced its expected outputs?
5a. Delivery of components and sub-systems (e.g., space-qualified modulators and demodulators, disk-drive, Ka-band RF chains, etc.) required for building the Cascade payload (OP01) 5a.1 Extent to which components and sub-components were developed, assembled and tested within schedule and budget.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
5a.2 The Cascade payload met all technical performance specifications, as determined by environmental tests.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
5b. Delivery of components and sub-systems required for building the Cascade portable ground terminals and Experimental Service Control Centre (OP01) 5b.1 Extent to which components and sub-systems were delivered within schedule and budget.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
5b.2 Cascade Service Control Centre and ground terminals meet all technical performance specifications (e.g., small size terminal with high bandwidth, multiple 320 Mbps channel Ka-band) required for demonstrating the Cascade service.
  • Document review (performance, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
5c. Design and manufacture of a SmallSat bus (OP02) 5c.1 Extent to which a SmallSat bus was developed, assembled, and tested in Canada within schedule and budget.
5c.2 The SmallSat bus meets all technical performance specifications (e.g., mechanical structure, solar panels, altitude control system), as determined by environmental tests.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
5c.3 The SmallSat bus meets all technical performance specifications (e.g., mechanical structure, solar panels, altitude control system), as determined by environmental tests.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
5d. Delivery of the eight scientific instruments and data handling unit required for building the ePOP payload (OP03) 5d.1 Extent to which the scientific instruments and data handling unit were developed, assembled and tested within schedule and budget.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from the University of Calgary.
5d.2 ePOP payload meets all technical performance specifications, as determined by environmental tests.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase)
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from the University of Calgary.
5e. On-Orbit orbit operations of the Cascade payload and demonstration of a digital satellite courier service (OP04) 5e.1 On-orbit Cascade payload is capable of demonstrating all the technologies and services required for possible follow-on pre-production satellites.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
5e.2 On-orbit Cascade payload and associated ground terminals are capable of picking up and delivering huge data packages (50 to 500 Gigabytes at a time) with delivery within a day to and from anywhere on Earth.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff
    • Representatives from MDA/Cascade Data Services.
5e.3 The Service Control Centre Prototype provides the necessary functionality for Cascade scheduling, ground terminal management, and back-channel data recovery control.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
5f. On-orbit operation of the CASSIOPE spacecraft, including demonstration of the small satellite bus capabilities required for effectively carrying both Cascade and ePOP payloads (OP05) 5f.1 Extent to which the on-orbit SmallSat bus operations meet the requirements for carrying both Cascade and ePOP payloads.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff
    • Representatives from MDA/Cascade Data Services
5g. On-orbit operations of the ePOP payload with its eight scientific instruments and the data handling unit required for building the ePOP payload (OP06) 5g.1 Extent to which on-orbit ePOP payload is capable of capturing and transmitting scientific data on plasma and atmospheric outflow processes in the polar ionosphere and upper atmosphere, as per the target performance specifications.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from the University of Calgary.
6. To what extent has CASSIOPE achieved its immediate outcomes?
6a. To what extent did CASSIOPE contribute to a reduction of the technology risks associated with the Cascade digital satellite courier service through an on-orbit demonstration? (IM01) 6a.1 Extent to which is successful on-orbit demonstration of the Cascade digital satellite courier likely to contribute to reducing the technology risks.
  • Document review (performance reports, activity tracking, review, progress and approval documents at/during each project phase).
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
6a.2 Opinions on the extent to which initial technology risks assessed as medium to high are either fully removed or rated as very low, as a result of the Demonstration (CX) phase.
  • Document review.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
6b To what extent did CASSIOPE contribute to improving the suitability of the first Canadian small satellite bus for the construction of future Cascade satellites through the successful operation of the CASSIOPE SmallSat bus? (IM05) 6b.1 Opinions on the likelihood that the SmallSat bus will be suitable for the construction of future Cascade satellites.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.

6c. To what extent did CASSIOPE contribute to the acquisition of scientific knowledge through ePOP data, specifically:

  • Knowledge of plasma dynamics in Earth's ionosphere; and
  • Enhanced understanding of the effect of plasma dynamics on radio transmissions, transport of gasses and Earth's climate? (IM06)
6c.1 Opinions on the extent to which data collected via ePOP is likely to facilitate the acquisition of scientific knowledge.
  • Document review.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from the University of Calgary.
    • Researchers, graduate students, research fellows.
7. To what extent has CASSIOPE achieved its intermediate outcomes?
7a. To what extent did CASSIOPE, particularly the Cascade component, contribute to maintaining the core manufacturing capabilities of the Canadian satellite communication industry? (IN02) 7a.1 Extent to which the Cascade component contributed to maintaining the core manufacturing capabilities of the Canadian satellite communication industry.
  • Document review.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
7a.2 Income and employment of the satellite communications companies participating in CASSIOPE are maintained at the level existing in 2003.
  • Document review.
  • Key informant interviews:
    • CSA managers and staff
    • Representatives from Communications Research Centre (Industry Canada)
    • Representatives from MDA/Cascade Data Services
7b. To what extent has the data supplied via ePOP to date is likely to contribute to an increased capability of models to predict space weather and its impact on radio transmissions? (IN04) 7b.1 Opinions on the extent to which the data supplied via ePOP are likely to contribute to an increased capability of models to predict space weather and its impact on radio transmissions.
  • Document review.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from the University of Calgary.
    • Researchers, graduate students, research fellows.
7c. To what extent has the data supplied via ePOP data is likely to contribute to increased security of radio transmissions, pipelines and electrical transmission systems through better prediction of space weather? (IN05) 7c.1 Extent to which the data supplied via ePOP is likely to contribute to improved prediction of space weather.
  • Document review:
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from the University of Calgary.
    • Researchers, graduate students, research fellows.
7c.2 Extent to which the data supplied via ePOP is likely to contribute to increased security of radio transmissions, pipelines and electrical transmission systems.
  • Document review.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from the University of Calgary.
    • Researchers, graduate students, research fellows.
8. To what extent has CASSIOPE achieved its ultimate outcomes?
8a. To what extent has CASSIOPE contributed to the expansion of Canada's space industry through the development of advanced technologies and equipment? (U01) 8a.1 Extent to which CASSIOPE has contributed to an expansion of Canada's space industry through development of advanced technologies and equipment.
  • Document review.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
8a.2 Extent to which CASSIOPE will continue to have a beneficial impact on the Canadian space industry over the coming five years.
  • Document review.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
8a.3 Growth in total income of companies participating in CASSIOPE and attributable to the mission.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
8b. To what extent has CASSIOPE contributed to the strengthening of the Canadian space industry with the development of a competitive prime contractor capacity and a team of effective suppliers to serve future missions? (U02) 8b.1 Extent to which CASSIOPE has contributed to the strengthening of Canada's space industry as a result of the development of a competitive prime contractor capacity and effective suppliers to serve future missions.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
8b.2 Extent to which CASSIOPE will continue to have a beneficial impact on the Canadian space industry over the coming five years.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
8b.3 Increase in domestic and export sales of the CASSIOPE industrial team attributable to participation in CASSIOPE.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
8c3 To what extent has CASSIOPE, specifically ePOP, contributed to the maintenance of the leadership role of Canadian scientists in leading-edge international research on space environment plasma dynamics and radio science? (U03) 8c.1 Extent to which ePOP is likely to contribute to the maintenance of the leadership role of Canadian scientists in research on space environment plasma dynamics and radio science.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from the University of Calgary.
    • Researchers, graduate students, research fellows.
8c.2 Extent to which ePOP is likely to have a beneficial impact on international research over the coming five years.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from the University of Calgary.
    • Researchers, graduate students, research fellows.

9. To what extent has CASSIOPE contributed to the Government of Canada Whole-of-Government Framework by:

  • Supporting the design and manufacture of world-class innovative technologies, products and/or services;
  • Providing training opportunities to many young scientists and engineers; and
  • Promoting Canadian industry abroad through exposure to international partnerships.
9.1 # of young scientists provided with training opportunities.
  • Key informant interviews:
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.
9.2 # of engineers provided with training opportunities.
  • Key informant interviews:
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.
9.3 # of international partnerships resulting from CASSIOPE.
  • Key informant interviews:
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.
9.4 Extent to which CASSIOPE has facilitated (or is likely to facilitate) the promotion of the Canadian space industry internationally.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary
10. Is the project design appropriate for achieving expected program results? 10.1 Clearly defined and understood governance structure, including program processes, roles, responsibilities and accountabilities.
  • Document review.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.
10.2 CSA resources/capacity directed towards CASSIOPE commensurate with expected results.
  • Document review.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.
10.3 Appropriateness of CASSIOPE activities, processes and governance structures.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from Communications Research Centre (Industry Canada).
11. Have there been any unintended (positive or negative) outcomes? 11.1 Presence/absence of unintended outcomes.
  • Document review.

12. Is the project undertaking activities and producing outputs in the most efficient manner?

  • How could the efficiency of the project's activities be improved?
  • Are there alternative, more efficient, ways of delivering the project?
12.1 Actual versus planned expenditures.
  • Financial analysis.
12.2 Cost of producing program outputs is as low as possible.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.
12.3 Possible improvements to the efficiency of CASSIOPE activities.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.
12.4 Possible alternatives, more efficient, ways of delivering program activities and outputs.
  • Key informant interviews:
    • CSA managers and staff.
    • Representatives from MDA/Cascade Data Services.
    • Representatives from the University of Calgary.
13. Is the project achieving its intended outcomes in the most economical manner? 13.1 Extent to which CASSIOPE's intended outcomes to date have been achieved at the least possible cost to the CSA.
  • Financial analysis.
13.2 Extent to which good value is being obtained with respect to the use of public funds.
  • Key informant interviews:
    • CSA managers and staff
    • Representatives from MDA/Cascade Data Services
    • Representatives from the University of Calgary.

Footnotes

Footnote 1

CAscade, SmallSat and IOnospheric Polar Explorer.

Return to footnote 1 referrer

Footnote 2

CAscade, SmallSat and IOnospheric Polar Explorer.

Return to footnote 2 referrer

Footnote 3

Industry Canada's Technology Partnership Canada (IC-TPC) executed a contribution agreement with MDA in March 2001 in the amount of $77 million. The contribution agreement was amended in September 2012 to reflect the de-scoping of the commercialization of the project. The total contract value was amended downwards to $48.6 million. This is described further in section 2.

Return to footnote 3 referrer

Footnote 4

Industry Canada's Technology Partnership Canada (TPC) executed a contribution agreement with MDA in March 2001 in the amount of $77 million. The contribution agreement between IC-TPC and MDA was amended in September 2012 to reflect the de-scoping of the commercialization of the project. The total contract value was amended downwards to $48.6 million. The TPC program was discontinued (i.e., stopped accepting new applications) in December 2006. The management of ongoing projects is being handled by IC's Industrial Technologies Office (ITO).

Return to footnote 4 referrer

Footnote 5

Reaching Higher: Canada's Interests and Future in Space, Aerospace Review, V.2, November 2012.

Return to footnote 5 referrer

Footnote 6

Aerospace Review, Volume 2, p. 3.

Return to footnote 6 referrer

Footnote 7

Mobilizing Science and Technology to Canada's Advantage, Industry Canada, May 2007.

Return to footnote 7 referrer

Footnote 8

CASSIOPE Monthly Progress Report September 2013.

Return to footnote 8 referrer

Footnote 9

Swarm is ESA's first constellation of satellites to advance understanding of how Earth works. Harnessing European and Canadian technology, the three Swarm satellites will measure precisely the magnetic signals that stem from Earth's core, mantle, crust and oceans, as well as its ionosphere and magnetosphere. By analyzing the different characteristics of the observed field, this state-of-the-art mission will lead to new insights into many natural processes, ranging from those occurring deep inside the planet to weather in space caused by solar activity. In turn, this information will yield a better understanding of why the magnetic field is weakening.

Return to footnote 9 referrer

Footnote 10

Note that MDA has transferred responsibility for operating the satellite for a period of 18 months to the University of Calgary. This is a contractual agreement between MDA and the U of C.

Return to footnote 10 referrer

Footnote 11

Hickling Arthurs Low, The State of the Canadian Space Sector, August 2012. Research report commissioned by the Aerospace Review.

Return to footnote 11 referrer