Mid-Term Review of Management and Performance of the Cassiope Contribution Program

Download the PDF version (306 KB)

Prepared by
Audit Services Canada

With collaboration from
Audit, Evaluation and Review Directorate

May 2007

Executive Summary

Context of Mid-Term Review

The CASSIOPE contribution program was authorized for a five year period on October 30, 2003. The CASSIOPE program amalgamates three program elements at the technology demonstration phase: it provides for the integration of two different payloads, the Cascade telecommunications payload and the Enhanced-Polar Outflow Probe (ePOP) scientific payload, on a single generic Canadian Small Satellite (SmallSAT) bus.

Two contribution agreements with recipients were developed by the Canadian Space Agency (CSA): one with MacDonald Dettwiler and Associates Ltd (MDA)/Cascade Data Services inc. (CDS) for the technology demonstration of the CASSIOPE mission, including the Cascade payload, and phases C and D (development and manufacturing) of the SmallSAT bus, the other with the University of Calgary for the ePOP payload.

The mid-term review assesses the relevance and adequacy of planning, design and management of the CASSIOPE contribution program, as well as progress made towards achieving expected results. It examines contribution agreements, and management processes and mechanisms ensuring program delivery and reporting on performance.

Findings of Mid-Term Review

Relevance Issue : Program Planning and Design

The CASSIOPE contribution program meets the CSA and Government of Canada priorities (technical, scientific and industrial). The CASSIOPE mission constitutes an ambitious integration of three different program elements that contribute to achieving the CSA's mandate in a context of resource limitation.

The contribution agreements with the two recipients constitute an adequate funding vehicle for the CASSIOPE program and an efficient mechanism to ensure limitation of government funding. Satisfactory progress towards meeting the expected technical results of the technology demonstration phase was achieved through the funding mechanism.

Effectiveness Issue : Program Management

The CSA program management office proactively manages technical aspects of the CASSIOPE contribution program. Effective management mechanisms and practices are implemented to ensure progress towards achieving expected technical results at the technology demonstration phase. The risk management mechanisms ensure that technical risks are effectively monitored and mitigated or reduced for all program elements.

The CSA disposes of sufficient authority and resources to ensure effective management of technical aspects of the CASSIOPE contribution program. The existing management mechanisms and practices provide the CSA with high level expertise in satellite communications and ensure effective coordination with the Communications Research Centre. They also ensure effective coordination between the CSA and each of the two recipients, as well as between the two contribution recipients.

However, the existing management mechanisms and practices do not ensure effective monitoring and reporting on commercial aspects of the Cascade technology, and do not provide for effective coordination with Technology Partnerships Canada on the feasibility of commercial aspects of the Cascade technology after completion of the technology demonstration phase. Considering the high financial risks associated with repayment of the $63M government contribution by MDA/CDS, corrective measures need to be implemented to improve the effectiveness of management of Cascade commercial aspects of the CASSIOPE contribution program.

Progress Issue : Program Performance

All three elements of the CASSIOPE contribution program have made significant progress towards meeting expected technical results of the technology demonstration phase. An updated assessment of the status of expected results (technical, commercial, scientific and industrial) identified in the Terms and Conditions for the program will only be possible after completion of the technology demonstration phase of the CASSIOPE mission.

As of March 2007, progress is insufficient with regards to attracting potential investors and potential clients for the Cascade technology at the commercialization phase. The CSA's approach is not appropriate to ensure progress towards meeting expected commercial results for the Cascade technology, in order to improve the likelihood of repayment of the $63M government contribution by MDA/CDS. Consequently, a proactive approach needs to be implemented to ensure progress towards meeting expected commercial results for the Cascade technology at the subsequent commercialization phase.

Conclusion and Recommendations of Mid-Term Review

The mid-term review demonstrates that effective management mechanisms and practices are implemented by the program management office in relation to technical aspects of the CASSIOPE contribution program at the technology demonstration phase, but not to commercial aspects of the Cascade technology.

In order to improve the effectiveness of management of commercial aspects of the Cascade technology, and thus the likelihood of repayment of the $63M government contribution by MDA/CDS, Audit Services Canada recommends that :

  1. The Canadian Space Agency require MacDonald Dettwiler Associates Ltd/Cascade Data Services inc. to finalize by the end of December 2007 a new and completely updated Cascade business plan detailing the major scenarios for attracting potential investors and penetrating potential markets for the Cascade technology.
  2. The Canadian Space Agency implement an ongoing monitoring and reporting mechanism, in coordination with Technology Partnerships Canada, on potential investors and potential markets for the Cascade technology.
  3. The Canadian Space Agency strengthen the program management office by adding on an ad hoc basis a specialized resource in financing and marketing in the space sector to proactively interact on an ongoing basis with the MacDonald Dettwiler and Associates Ltd/Cascade Data Services inc. business development team.

Table of Contents

List of Acronyms

  • ASC: Audit Services Canada
  • CDS: Cascade Data Services inc.
  • CRC: Communications Research Centre Canada
  • CSA: Canadian Space Agency
  • ePOP: Enhanced-Polar Outflow Probe
  • MDA: MacDonald Dettwiler and Associates Ltd
  • PAD: Project Approval Document
  • PRAB: Project Review Advisory Board
  • RMAF: Results-Based Management and Accountability Framework
  • SmallSAT: Small Satellite
  • TPC: Technology Partnerships Canada

1. Introduction

1.1 Background and Mandate

The CASSIOPE contribution program was authorized for a five year period on October 30, 2003. The CASSIOPE program amalgamates three program elements at the technology demonstration phase : it provides for the integration of two different payloads, the Cascade telecommunications payload and the Enhanced-Polar Outflow Probe (ePOP) scientific payload, on a single generic Canadian Small Satellite (SmallSAT) bus. The CASSIOPE mission is described as follows in the contribution program poster:

"Canada's first hybrid satellite, CASSIOPE, is built on a new multi-purpose platform designed through the CSA's SmallSAT program. This new bus design is adaptable to a wide range of missions, including science, technology, earth observation, geological exploration and telecommunications. Cascade, the telecommunications component of the mission, will demonstrate a new wordwide, high-speed, and large volume communications service. The scientific component of the mission, ePOP, will study plasma and atmospheric outflows in the polar regions and the wave generation, particle interaction and propagation associated with these outflows." Footnote 1

Two contribution agreements with recipients were developed by the Canadian Space Agency (CSA) : one with MacDonald Dettwiler and Associates Ltd (MDA)/Cascade Data Services inc. (CDS) for the technology demonstration of the CASSIOPE mission, including the Cascade payload, and phases C and D (development and manufacturing) of the SmallSAT bus, the other with the University of Calgary for the ePOP payload. The two contribution agreements are best summarized in the Submission to the Project Review Advisory Board (PRAB) – the Executive Committee mechanism responsible for program approval – dated September 10, 2003.

"The CSA contribution to Cascade Data Services for the Technology Demonstration consists of the development, deployment and on-orbit demonstration of a single experimental satellite (including the development of a Canadian bus suitable for future CSA satellites) and ground systems. This phase will reduce technical risks, non-recuring engineering and construction timelines for follow-on pre-production satellites, as well as demonstrate the viability of the service to prospective users and investors. The Technology Demonstration phase includes the detailed design, manufacture and test of a SmallSAT bus to carry the Cascade and ePOP payloads.

The CSA contribution to the University of Calgary for the ePOP mission, a high priority Canadian scientific initiative led by the University of Calgary, provides for the development of eight scientific instruments to explore plasma and atmospheric outflow processes in the polar ionosphere and upper atmosphere. Six of the eight instruments will be Canadian-made; the US Naval Research Labs and Japan's Institute for Space and Astronomical Sciences will respectively provide two at no cost." Footnote 2

The CSA established a program management office within the Space Programs Branch to provide management of the contribution agreements and coordination of the CASSIOPE mission, with delegated authority to Communications Research Centre Canada (CRC) for the development and demonstration of the Cascade payload program element.

The Terms and Conditions for the program stipulate that the CSA must undertake a mid-term review of the CASSIOPE contribution program, in line with the annexed Results-Based Management and Accountability Framework (RMAF). It is in that context that the CSA mandated Audit Services Canada (ASC) to conduct the mid-term review of management and performance of the CASSIOPE contribution program.

1.2 Scope of Mid-Term Review

The mid-term review assesses the relevance and adequacy of planning, design and management of the CASSIOPE contribution program, as well as progress made towards achieving expected results. It examines contribution agreements, and management processes and mechanisms ensuring program delivery and reporting on performance; when necessary, it recommends management practices needed to improve program delivery and performance. This approach is congruent with the Terms and Conditions stating that the review should focus on management issues and explain if progress towards the achievement of outcomes is occurring.

The mid-term review of management and performance of the CASSIOPE contribution program examines three issues:

  • The relevance issue (program planning and design) addresses whether the contribution program responds adequately to the CSA and Government of Canada priorities.
  • The effectiveness issue (program management) addresses whether the delivery mechanisms of the contribution program ensure progress towards achieving expected results.
  • The progress issue (program performance) addresses whether the contribution program, at mid-term, has made progress in achieving expected results.

1.3 Methodology of Mid-Term Review

The mid-term review of the CASSIOPE contribution program was conducted in five consecutive phases. The list of documents that were reviewed by ASC is included further in the report.

  • Phase 1 consisted in reviewing official CASSIOPE program documents and CSA corporate documents, and in developing a framework for the mid-term review identifying questions, criteria and information sources for each of the three issues. Phase 1 was conducted with the constant collaboration of the CASSIOPE Program Integrator and in consultation with the Audit, Evaluation and Review Directorate. The framework for mid-term review was approved by the Project Management Processes and Special Projects Directorate of Space Programs Branch.
  • Phase 2 consisted in conducting direct interviews with 14 managers and professionals from the Space Programs Branch, the Space Technologies Branch and the Space Science Branch. ASC reviewed numerous project documents that were accessed through Livelink, the CSA document management system.
  • Phase 3 consisted in conducting direct interviews with officers of two other government departments associated with the CASSIOPE contribution program, the CRC and Technology Partnerships Canada (TPC), as well as with the ePOP Project Leader at the University of Calgary. A teleconference has held with the CASSIOPE Program Manager at MDA and the Manager Business Development at CDS. ASC also reviewed various relevant documents transmitted by some interviewees. Phase 3 was conducted with the collaboration of the CASSIOPE Deputy Program Manager.
  • Phase 4 consisted in writing the draft version of the report by summarizing the findings of the mid-term review and formulating the conclusion and recommendations, as well as validating the findings and discussing the conclusion and recommendations with CSA senior management.
  • Phase 5 consisted in writing the final version of the report and including the management action plan developed by the CSA to implement recommendations.

ASC wishes to thank the managers and professionals of the CSA, CRC, TPC, the University of Calgary and MDA/CDS for their availability, their proactivity and the relevance of their comments at all phases of the mid-term review.

2. Findings of Mid-Term Review

This section summarizes the main findings of the mid-term review of management and performance of the CASSIOPE contribution program concerning three issues : the relevance issue (program planning and design), the effectiveness issue (program management), and the progress issue (program performance).

2.1 Relevance Issue : Program Planning and Design

The relevance issue examines whether program planning and design respond adequately to CSA and Government of Canada priorities. The framework for mid-term review identifies four questions pertaining to the relevance issue, which are addressed in the two following sub-sections: objectives and expected results, and financing of contribution program.

2.1.1 Objectives and Expected Results

The overall objective of the contribution program was clearly explained in the CASSIOPE mission Project Approval Document (PAD) presented to PRAB on September 10, 2003:

"The state of Canada's satellite communication industry remains a source of concern. The industry has been severely affected by the collapse of the international telecommunications sector, its key source of export revenues... The CASSIOPE Mission will sustain highly qualified jobs and manufacturing capability in the Canadian space industry (which has been largely created by the federal government through procurement over the last thirty years) and will help overcome difficulties due to the severe economic slump faced by the World communication sector." Footnote 3

The ePOP PAD revised as of February 15, 2005 also states that the ePOP mission "is a high priority Canadian scientific initiative... (and) will be the only space-borne instrument platform for Canada's space-environment community in the near future." Footnote 4

The CASSIOPE mission remains a priority for the CSA. The Report on Plans and Priorities for the 2006-2007 Estimates indicates that one of the main focus of the Satellite Communications thrust (which generates 74,8% of Canadian space sector's revenues) of the Canadian Space Strategy is "the development of a high-speed, high-capacity space messaging experimental payload called Cascade, of interest to resource exploration firms, industrial clients, and remote research communities." Footnote 5

The CASSIOPE contribution program meets the CSA and Government of Canada priorities (technical, scientific and industrial). The CASSIOPE mission constitutes an ambitious integration of three program elements (the Cascade payload, the ePOP payload and the SmallSAT bus) that contribute to achieving the CSA's mandate in a context of resource limitation.

The key expected results and main objectives of the three elements of the CASSIOPE contribution program, as identified in the Terms and Conditions of the program, are fully understood by CSA officers and contribution recipients. The key expected results for the Cascade payload and SmallSAT bus, as well as for the ePOP payload are described as follows in the Terms and Conditions:

"(Cascade Payload / SmallSAT Bus) The timely development of leading-edge satellite communications and bus technologies to enhance Canadian industry capabilities and build a new global advanced electronic data transmission service in Canada.

(ePOP Payload) To allow Canadian and international scientists to enhance the understanding of the plasma dynamics in the ionosphere, which would contribute to better predicting space weather, transporting gases and reducing distortion of radio transmissions." Footnote 6

The RMAF logic model annexed to the Terms and Conditions identifies more detailed expected results for the CASSIOPE contribution program, but the performance indicators provided by the accompanying performance measurement strategy are not operational. The Terms and Conditions do not define operational performance indicators to monitor progress in program implementation. Therefore, the program management office resorted to established project management mechanisms as performance indicators, and used schedule and budget criteria to monitor milestone achievements.

2.1.2 Financing of Contribution Program

The Terms and Conditions approved costs of $63M for the Cascade payload and phases C and D (development and manufacturing) of the SmallSAT bus, of $11M for the ePOP payload and of $6M for management and coordination by the program management office from Fiscal Year 2003-2004 to Fiscal Year 2007-2008. Table 1 on next page details by Fiscal Years the costs approved by TBS for the CASSIOPE contribution program.

Table 1 – Costs Approved by Terms and Conditions for CASSIOPE Contribution Program ($Millions)

Program Elements Fiscal Years Total
2003-04 2004-05 2005-06 2006-07 2007-08
Cascade Payload/SmallSAT Bus $5.7 $14.3 $25.0 $16.5 $1.5 $63.0
ePOP Payload $0.6 $3.3 $4.5 $2.4 $0.2 $11.0
Program Management Office $0.4 $1.5 $1.5 $1.5 $1.1 $6.0

Source: Canadian Space Agency (2003), Terms and Conditions - Establishment of a Contribution Program to Support the Joint Cascade Technology Demonstration/Enhanced-Polar Outflow Probe Small Satellite (CASSIOPE) Mission, October 30 2003, pages 2-3.

The December 2003 contribution agreement with MDA/CDS made provisions for a repayable contribution of $63M; two amendments to the contribution agreement (in February 2006 and February 2007) modified cashflow distribution but not the amount of contribution. The February 2004 contribution agreement with the University of Calgary made provisions for a non-repayable contribution of $10 321 000; two amendments to the contribution agreement (in September 2005 and November 2006) modified cashflow distribution and the second amendment adjusted the non-repayable contribution to $11M.

The total government financial assistance to MDA/CDS at the technology demonstration phase is $111.6M, of which the CSA contributes $63.0M (55.7%) and TPC $48.6M (44.3%). Hence, since the beginning of the technology demonstration phase with the TPC contribution in April 2000, cumulative financing from MDS/CDS is 11.8% ($14.9M/$126.5M) of eligible costs of the project. Table 2 details (as stated in the Terms and Conditions) the total government financial assistance to MDS/CDS for the Cascade payload and phases C and D of the SmallSAT bus at the technology demonstration phase.

Table 2 – Total Government Financial Assistance at Technology Demonstration Phase ($Millions)

Technology Demonstration Phase Total Eligible Costs Industry Investment in Eligible Costs TPC Contribution CSA Contribution Total Government Contribution
Program Definition Stage $5.0 $3.0 $2.0 $0 $2.0
Technology Demonstration Stage $113.1 $4.9 $45.2 $63.0 $108.2
Early Entry Strategy Stage $8.4 $7.0 $1.4 $0 $1.4
Total $126.5 $14.9 $48.6 $63.0 $111.6 (88.2%)

Source: Canadian Space Agency (2003), Terms and Conditions - Establishment of a Contribution Program to Support the Joint Cascade Technology Demonstration/Enhanced-Polar Outflow Probe Small Satellite (CASSIOPE) Mission, October 30 2003, page 36.

The combined contribution rate of the CSA and TPC to MDA/CDS is 58.7% of eligible costs for the technology demonstration and commercialization phases and 88.2% of eligible costs for the sole technology demonstration phase of the CASSIOPE program. The combined contribution rate for the technology demonstration phase far exceeds the 75% threshold for total government financial assistance to an industrial research and development project. Because of the likelihood that the commercialization phase would not materialize, the Minister of Industry Canada signed a Memorandum (annexed to the Terms and Conditions) granting approval to an increase of total government financial assistance to a maximum of 90% of eligible costs for the technology demonstration phase.

The increase to a maximum of 90% of total government financial assistance at the technology demonstration phase was a bold decision by senior management. The overall program risk for the CASSIOPE mission was rated as high by both the CSA and TPC in 2003, as were the Cascade payload and SmallSAT bus program elements. The September 2003 CASSIOPE mission PAD stated the following concerning risk assessment :

"The overall project risk is rated high... The financial risk is a key concern, since MDA does not have all of the internal financial resources to fund its share of total project costs and has yet to secure the required financing. MDA's strategy is to use the CSA's contribution to retire technical risk during the technology demonstration phase. Should this phase fall short of meeting its pre-set technical objectives and fail to draw sufficient users to attract private equity, the project would come to a halt. Industry Canada (TPC and CRC) and CSA consider the occurrence of that risk to be significant." Footnote 7

Considering that materialization of the commercialization phase is undocumented as of March 2007, assessment of potential returns will only be possible after completion of a successful technology demonstration phase. It is foreseen that all targeted scientific returns (from the ePOP payload) will then have been achieved, but only some targeted industrial and technological returns (from the Cascade payload and the SmallSAT bus). However, other targeted industrial and technological returns will only be achieved at the commercialization phase; the CSA should thus closely monitor preparation for the commercialization phase, as is explained further in the report.

Because of the high level of risk and high rate of government financial assistance, CSA senior management decided to design the CASSIOPE contribution program as if it were a contracted-out project. The hands-on approach for the management and coordination of technical aspects of the program allowed the CSA to exercise the due diligence required by the integration of three different program elements and the imposed partnership between industry and academia. It also allowed the CSA to alleviate the underfunding for the development of the generic SmallSAT bus by providing expertise to the supplier.

The contribution agreements with the two recipients constitute an adequate funding vehicle for the CASSIOPE program and an efficient mechanism to ensure limitation of government funding. Satisfactory progress towards meeting the expected technical results of the technology demonstration phase was achieved through the funding mechanism.

2.2 Effectiveness Issue : Program Management

The effectiveness issue examines whether the delivery mechanisms of the contribution program ensure progress towards achieving expected results. The framework for mid-term review identifies seven questions pertaining to the effectiveness issue, which are addressed in the two following sub-sections: management of technical aspects of the program, and management of commercial aspects of the program. The mid-term review does not examine financial management of the program, as financial issues are addressed in the 2007 internal audit of the CASSIOPE contribution program.

2.2.1 Management of Technical Aspects of Program

The CSA contributes significant financial resources to management of the contribution agreements and coordination of the CASSIOPE mission, due to the integration of three different program elements. In this context, the program management office ensures effective management of technical aspects of the CASSIOPE program, based on a proactive approach of integrated project team. The program management office derives its authority from the relevance of technical recommendations and solutions offered to the two recipients and their sub-contractors.

The program management office exercises leadership to ensure collaboration and technical coordination between the two contribution recipients, MDA/CDS and the University of Calgary.

In the course of the mid-term review, the ePOP Project Leader from the University of Calgary expressed his satisfaction with the relationship and collaboration process that have been gradually built since 2004 with MDA/CDS, the prime contractor of the CASSIOPE mission.

The CSA has delegated authority to CRC for the development and demonstration of the Cascade payload, including the ground segment, in a Memorandum of Understanding signed on June 25, 2004. CRC, an Agency of Industry Canada, constitutes the Government of Canada's centre of expertise in satellite communications. CRC acts as the Cascade project manager within the CSA program management office and ensures technical coordination with MDA/CDS and its sub-contractors for the Cascade payload, but was not delegated financial authority to approve claims for progress payments.

The program management office established the mechanisms and practices required by the technical risks and complexity of the CASSIOPE contribution program. Firstly, the program management office implemented management mechanisms that are in line with the CSA's Project Approval and Management Framework, particularly the following : work breakdown structure and work packages, responsibility assignment matrix, project schedule, project budget, and Risk Information Assessment System. Secondly, the project management office implemented its integrated project team approach for technical coordination by holding weekly teleconferences with recipients engineers and managers (and their sub-contractors) for each of the three program elements. Thirdly, the program management office established reporting on technical aspects of the program on a monthly (Program Progress Reports, Director General Program Status Reviews) and quarterly (Quarterly Program Reviews) basis. All reporting documents are easily accessed through Livelink, the CSA document management system.

The Risk Information Assessment System was implemented at the planning and design stage of the contribution program, to monitor the risk assessment of the Risk-Based Audit Framework annexed to the Terms and Conditions. This risk monitoring and reduction mechanism encompasses cost, schedule, technical and programmatic risks. The resulting risk matrix, based on likelihood and consequence of occurrence, is updated and reported to senior management on a monthly basis. The identified risks are monitored until they are deemed to be closed.

As of February 2007, 29 CASSIOPE program risks (including SmallSAT bus and ePOP payload risks) have been closed (7 cost risks, 5 schedule risks, 11 technical risks and 6 programmatic risks). Fifteen risks are still open, of which 3 are rated as high risks, 6 as medium risks, and 6 as low risks by the risk matrix. Table 3 identifies the CSA rating of open risks for the CASSIOPE contribution program.

Table 3 – Rating of Open Risks for CASSIOPE Contribution Program (as of February 2007)

Rating Category of Risks
Cost Schedule Technical Programmatic
High Risks   Launch Delay  
  • Contribution Repayment to Government
  • ePOP Payload
Medium Risks
  • Cost Risk
  • Schedule Risk-Program Sustainment
  • Mission Operation System
 
  • Generic SmallSAT Bus
  • Cascade Payload
  • SmallSAT Infrastructure
Low Risks      
  • Programmatic Risk
  • Communication Failure with Stakeholders
  • CSA Level of Support Insufficient
  • Sustainability of Canadian Space Industry
  • Loss of Spacecraft During Launch
  • Mission Failure

Source: Canadian Space Agency (2007), CASSIOPE/ePOP Contribution Program - Director General Program Status Review (February 20 2007), page 20.

The CSA program management office effectively manages technical aspects of the CASSIOPE contribution program. Effective management mechanisms and practices are implemented to ensure progress towards achieving expected technical results at the technology demonstration phase. The risk management mechanisms ensure that technical risks are effectively monitored and mitigated or reduced for all program elements.

The CSA disposes of sufficient authority and resources to ensure effective management of technical aspects of the CASSIOPE contribution program. The existing management mechanisms and practices provide the CSA with high level expertise in satellite communications and ensure effective coordination with CRC. They also ensure effective coordination between the CSA and each of the two recipients, as well as between the two contribution recipients.

2.2.2 Management of Commercial Aspects of Program

The mid-term review demonstrates weaknesses in management by the CSA of commercial aspects of the program. Questions arising from the commercial aspects of the Cascade technology are not new. An evaluation report commissioned by the CSA in 2003 came to the conclusion that "there seems to be a lack of substantiation or compelling evidence for the assumptions on the addressable market." Footnote 8

MDA/CDS has been without a documented valid Cascade business plan since the start of Fiscal Year 2006-2007. MDA/CDS announced at the Senior Management Meeting held on April 12, 2006 that investments by Canadian companies at the technology demonstration phase had not materialized, as expected in the original 2003 Cascade business plan. MDA/CDS then announced at the Quarterly Program Review held on May 03, 2006 that a decision had been made to revise the Cascade business plan given significant changes in the marketplace.

As of March 2007, the documented Cascade business plan consists of two pages of a PowerPoint presentation at the Quarterly Program Review held on March 08, 2007. In the course of the mid-term review, it was stated by MDA/CDS managers that identification of a market base is problematic and that four possible permutations are presently considered for commercialization of the Cascade technology, depending on types of clients : (1) the flight of the Cascade payload on a MDA/CDS spacecraft for one customer; (2) the flight of the Cascade payload on a MDA/CDS spacecraft for several customers; (3) the flight of the Cascade payload on another spacecraft; (4) the sale of sub-systems of the Cascade payload. It was also stated by MDA/CDS managers that it would not be relevant to table an updated Cascade business plan before the launch of the spacecraft.

Although repayment of the $63M government contribution by MDA/CDS is rated as a high risk, monitoring and reporting on commercial aspects of the Cascade technology do not occur on an ongoing basis, as with technical aspects of the CASSIOPE program. Because there is no effective monitoring of commercial aspects, the CSA does not require MDA/CDS to effectively substantiate the commercial potential of the Cascade technology at the commercialization phase. Consequently, the CSA does not have in file updated and documented information on two major aspects of any credible business plan, potential investors and potential markets.

Furthermore, coordination with TPC (the other government department involved in financing the development and commercialization of the Cascade technology) is not sufficiently sustained to ensure adequate monitoring of the feasibility of commercial aspects of the Cascade technology, and of the likelihood of repayment of the government contribution by MDA/CDS. Coordination with TPC does not occur on an ongoing basis and consists largely of contacts and exchanges during Quarterly Program Reviews.

The existing management mechanisms and practices do not ensure effective monitoring and reporting on commercial aspects of the Cascade technology, and do not provide for effective coordination with TPC on the feasibility of commercial aspects of the Cascade technology after completion of the technology demonstration phase. Considering the high financial risks associated with repayment of the $63M government contribution by MDA/CDS, corrective measures need to be implemented to improve the effectiveness of management of Cascade commercial aspects of the CASSIOPE contribution program.

2.3 Progress Issue: Program Performance

The progress issue examines whether the contribution program, at mid-term, has made progress in achieving expected results. The framework for mid-term review identifies five questions pertaining to the progress issue, which are addressed in the two following sub-sections: progress towards expected results, and repayment of the government contribution.

2.3.1 Progress Towards Expected Results

a) Technical Results

The schedule for reaching the original targeted milestones has been delayed by 18 months as of March 2007. In particular, delays have been caused by difficult negotiations between the prime contractor and its sub-contractors, delays in the development of the SmallSAT bus technology, delivery problems due to the tightening of ITAR requirements in the United States, as well as problems related to the development of the Data Storage Unit and the change of launch vehicle.

The original targeted milestones were set in the contribution agreements with MDA/CDS (in December 2003) and with the University of Calgary (in February 2004), and were later postponed in two amendments to the contribution agreements with both recipients. The latest estimated milestones for the ePOP payload are still those of the second amendment to the contribution agreement with the University of Calgary, but will in effect have to be adjusted to those of the CASSIOPE mission that have again been postponed as of March 2007. Table 4 details the original targeted milestones and the latest estimated milestones as of March 2007 for MDA/CDS (CASSIOPE Mission) and the University of Calgary (ePOP Payload).

Table 4 – Milestones for CASSIOPE Contribution Program at Technology Demonstration Phase

MDA/CDS Milestones (CASSIOPE Mission) Latest Estimated Milestones (March 2007) Original Targeted Milestones
1. Kick-Off (Completed) December 2003 November 2003
2. System Requirements Review (Completed) May 2004 February 2004
3. Preliminary Design Review (Completed) December 2004 July 2004
4. Qualification Status Review (Completed) March 2005 January 2005
5. Critical Design Review (Completed) March 2006 September 2005
6. Test Readiness Review September 2007 April 2006
7. Flight Readiness Review April 2008 October 2006
8. Launch Readiness Review June 2008 January 2007
9. Commissioning Completion Review August 2008 March 2007
10. Operation Completed Review October 2009 March 2008
University of Calgary Milestones (ePOP Payload) Latest Estimated Milestones (March 2007) Original Targeted Milestones
1. Completion of ePOP Preliminary Design Review (Completed) December 2004 March 2004
2. Completion of ePOP Critical Design Review (Completed) July 2005 September 2004
3. DHU Simulator Delivery (Completed) April 2005 March 2005
4. Completion of Procurement of Booms (Completed) March 2006 September 2005
5. DHU Delivery to Spacecraft for Integration December 2006 March 2006
6. Completion of Pre-Shipment Review March 2007 June 2006
7. Completion of Spacecraft Environmental Tests September 2007 October 2006
8. Completion of Launch Readiness Review October 2007 January 2007
9. Completion of Commissioning Completion Review March 2008 March 2007
10. Completion of 6 Months of Science Operation September 2008 September 2007
11. Completion of 12 Months of Science Operation March 2009 March 2008

Validated with the CSA CASSIOPE Deputy Program Manager.

The technical risks associated with existing milestones have been monitored and reduced as engineering work progresses. Interviews conducted with CSA managers and professionals enable the mid-term review to identify three outstanding technical risks as of March 2007.

The higher outstanding technical risk for the CASSIOPE mission is the potential of launch delay of the spacecraft, which relates to the launch vehicle chosen by MDA/CDS. A change of launch vehicle was announced by the prime contractor at the Critical Design Review meeting in March 2006, for which the CSA requested the implementation of a risk mitigation plan. The CSA's monitoring actions in relation to the change of launch vehicle are well documented.

Two other outstanding technical risks as of March 2007 concern the SmallSAT bus and the Cascade payload. The Command and Data Handling is the only remaining component of the SmallSAT bus offering significant technical challenges, while the Data Storage Unit is the only remaining component of the Cascade payload offering significant technical challenges. All components of the ePOP payload are in the process of being delivered.

The University of Calgary has met the prerequisites to ensure that scientific data generated by the ePOP payload during the technology demonstration phase are disseminated within the scientific community. The University of Calgary has assembled an ePOP Science Team that holds biannual science meetings and constitutes a network of stakeholders from at least nine Canadian universities : Calgary, Alberta, Athabasca, New Brunswick, Saskatchewan, Western Ontario, York, Victoria and Royal Military College. The CSA's Space Science Branch is also active in the ePOP Science Team. The Science Operations Centre (at the University of Calgary) and the Space Science Data Portal (at the University of Alberta) are the mechanisms that will ensure dissemination of data generated by the ePOP payload.

Considering that materialization of the commercialization phase is undocumented as of March 2007, assessment of the feasibility of meeting expected results will only be possible after completion of a successful technology demonstration phase. Firstly, it is foreseen that it will have resulted in a space qualified generic Canadian multi-purpose platform (SmallSAT bus). Secondly, it is foreseen that it will have generated leading-edge scientific data on space weather that could be disseminated in the scientific community (ePOP payload). Thirdly, it is foreseen that it will have created spin-off opportunities for sub-systems manufactured by MDA/CDS or sub-contractors; it will also have generated new technological knowledge in telecommunications and contributed to maintaining Canada's position in satellite communications.

All three elements of the CASSIOPE contribution program – the SmallSAT bus, the Cascade payload (including the ground segment), and the ePOP payload – have made significant progress towards meeting expected results for technical aspects of the technology demonstration phase. An updated assessment of the status of expected results (technical, commercial, scientific and industrial) identified in the Terms and Conditions will only be possible after completion of the technology demonstration phase of the CASSIOPE mission.

b) Industrial Results

Audit Services Canada is of the opinion that many objectives and expected results of the CASSIOPE contribution program – as identified in the RMAF annexed to the Terms and Conditions – cannot be reviewed before completion of the technology demonstration phase. Our framework for mid-term review identifies all questions for which assessment of progress made towards achieving certain objectives and expected results can be conducted at the mid-term phase of the program.

Concerning the expected industrial results of the CASSIOPE program, the RMAF annexed to the Terms and Conditions states that one intermediate outcome is to maintain the core manufacturing capabilities of the Canadian satellite communication industry. When the Terms and Conditions were developed in 2003, it was estimated (but not determined as a target) that the technology demonstration phase would produce 180 persons years of employment spread over a three year period (and thus 60 persons/year), and that the estimated regional distribution of contracts would have to be modified once the supplier of the SmallSAT bus was chosen.

Data collected by the CSA indicate that the CASSIOPE program has exceeded the original estimated industrial results. It is presently estimated that approximately 85 highly qualified direct jobs have been sustained during each of the three preceding years (2004, 2005 and 2006) : 25 direct jobs at MDA/CDS headquarters and various affiliates, 30 direct jobs at Bristol Aerospace, 20 direct jobs at major sub-contractors and 10 direct jobs at the University of Calgary and its network of Canadian universities. The CASSIOPE program has thus contributed to maintain the core manufacturing capabilities of the industry. In particular, it has enabled Bristol Aerospace to maintain its space division and major sub-contractors to develop sub-systems that could eventually be commercialized.

The present regional distribution of contracts is documented in biannual sub-contractor reports presented to the CSA by MDA/CDS and the University of Calgary. Every Canadian region has benefited from returns of the differents elements of the CASSIOPE program : The Cascade payload has generated regional returns in Ontario, Quebec and Atlantic; the SmallSAT bus has generated regional returns in the Prairies; the ePOP payload has generated regional returns in all regions. Finally, project management and operation activities have generated regional returns in Bristish Columbia and the Prairies.

2.3.2 Repayment of Government Contribution

Repayment of the $63M government contribution by MDA/CDS is rated as a high risk (in terms of likelihood and consequence of occurrence) by the CSA Risk Information Assessment System. TPC officers also rate repayment of government contribution as a very high risk. They estimate that the risk is primarily a financial one, for which reason financing of the commercial mission should precede sales for use of the Cascade payload capacity. However, in the course of the mid-term review, discussions with MDA/CDS managers put in perspective the fact that the present Cascade business strategy is primarily aimed at prospecting the market for business opportunities rather than securing financing from private investors.

Article 6.3 of the contribution agreement with MDA/CDS identifies the Launch Readiness Review, which is presently targeted for June 2008, as a milestone for MDA/CDS to demonstrate results towards financing the Cascade commercialization phase :

"It is understood and agreed that the Recipient shall, at the time of the Launch Readiness Review, show to the satisfaction of the Minister that the Recipient has generated investment by private investors, or commitment to invest by private investors, which is sufficient to allow the Recipient to fully implement the Early Entry stage of the Project. Should the Minister not be satisfied, at the time of the Launch Readiness Review, that the Recipient has generated investment by private investors, or commitment to invest by private investors, which is sufficient to allow the Recipient to fully implement the Early Entry stage of the Project, the Minister shall have the right to adjust future payments of the contribution to such extent that may prove necessary, so the total amounts paid by the Government of Canada no longer exceed 90% of the Eligible Costs already incurred by the Recipient, until such time that the Recipient has generated investment by private investors, or commitment to invest by private investors, which is sufficient to the satisfaction of the Minister to allow the Recipient to fully implement the Early Entry stage of the Project." Footnote 9

There is currently no specialized resource in the CSA program management office that is empowered to interact with the MDA/CDS business development team on questions pertaining to mission financing and technology marketing within the space industry. To reach the Launch Readiness Review milestone and until completion of the technology demonstration phase, which is presently targeted for October 2009, there is a need for the CSA to proactively participate in commercial problem solving with MDA/CDS, as is successfully done for technical aspects of the CASSIOPE contribution program.

As of March 2007, progress is insufficient with regards to attracting potential investors and potential clients for the Cascade technology at the commercialization phase. The CSA's approach is not appropriate to ensure progress towards meeting expected commercial results for the Cascade technology, in order to improve the likelihood of repayment of the $63M government contribution by MDA/CDS. Consequently, a proactive approach needs to be implemented to ensure progress towards meeting expected commercial results for the Cascade technology at the subsequent commercialization phase.

3. Conclusion of Mid-Term Review

In relation to the relevance issue, the mid-term review establishes that the contribution agreements with MDA/CDS and the University of Calgary constitute an adequate funding vehicle for the CASSIOPE contribution program and an efficient mechanism to ensure limitation of government funding, in the context of the integration of three different program elements and the risk associated with the increase to a maximum of 90% of total government financial assistance at the technology demonstration phase.

For the effectiveness issue, the mid-term review demonstrates that effective management mechanisms and practices are implemented by the program management office in relation to technical aspects of the CASSIOPE contribution program at the technology demonstration phase, but not to commercial aspects of the Cascade technology.

Considering the progress issue, the mid-term review assesses that all three CASSIOPE program elements have made significant progress towards meeting expected technical results at the technology demonstration phase. However, progress is insufficient towards meeting expected commercial results and securing repayment of the government contribution at the subsequent commercialization phase.

Corrective measures should be implemented to improve the effectiveness of management of commercial aspects of the Cascade technology, and thus the likelihood of repayment of the $63M government contribution by MDA/CDS.

4. Recommendations

In order to improve the effectiveness of management of commercial aspects of the Cascade technology, and thus the likelihood of repayment of the $63M government contribution by MacDonald Dettwiler and Associates Ltd/Cascade Data Services inc., Audit Services Canada recommends that :

  1. The Canadian Space Agency require MacDonald Dettwiler Associates Ltd/Cascade Data Services inc. to finalize by the end of December 2007 a new and completely updated Cascade business plan detailing the major scenarios for attracting potential investors and penetrating potential markets for the Cascade technology.
  2. The Canadian Space Agency implement an ongoing monitoring and reporting mechanism, in coordination with Technology Partnerships Canada, on potential investors and potential markets for the Cascade technology.
  3. The Canadian Space Agency strengthen the program management office by adding on an ad hoc basis a specialized resource in financing and marketing in the space sector to proactively interact on an ongoing basis with the MacDonald Dettwiler and Associates Ltd/Cascade Data Services inc. business development team.

List of Documents

Canadian Space Agency (2007), CASSIOPE/ePOP - Science, Technology and Programs Program Status Review (March 02 2007), 6 pages.

Canadian Space Agency (2007), CASSIOPE Contribution Program Poster as of February 28, 2007.

Canadian Space Agency (2007), CASSIOPE/ePOP Contribution Programs - Director General Program Status Review (February 20 2007), 26 pages.

Canadian Space Agency (2007), Amendment Number Two to the Contribution Agreement with Cascade Data Services inc. and MacDonald Dettwiler and Associates Ltd Concerning the CASSIOPE Mission, February 1st 2007, 4 pages.

Canadian Space Agency (2007), CASSIOPE Contribution Program - Progress Report January 2007, January 31 2007, 19 pages.

Canadian Space Agency (2006), Amendment Number Two to the Contribution Agreement with the Governors of the University of Calgary Concerning Development and Flight of an Enhanced - Polar Outflow Probe, November 1st 2006, 5 pages.

Canadian Space Agency (2006), Vérification du programme de contribution CASSIOPE. Termes de références, Octobre 2006, 5 pages.

Canadian Space Agency (2006), Report on Plans and Priorities. 2006-2007 Estimates, 71 pages and 13 pages.

Canadian Space Agency (2006), Performance Report for the Period Ending March 31 2006, 118 pages and 17 pages.

Canadian Space Agency (2006), Amendment Number One to the Contribution Agreement with Cascade Data Services inc. and MacDonald Dettwiler and Associates Ltd Concerning the CASSIOPE Mission, February 1st 2006, 5 pages.

Canadian Space Agency (2005), Amendment Number One to the Contribution Agreement with the Governors of the University of Calgary Concerning Development and Flight of an Enhanced - Polar Outflow Probe, September 1st 2005, 7 pages.

Canadian Space Agency (2005), Enhanced - Polar Outflow Probe - Project Approval Document, revised February 15 2005, 24 pages.

Canadian Space Agency (2004), Contract no 9F028-044404/001/MTB with Bristol Aerospace Limited for a SmallSAT Infrastructure, December 23 2004, 11 pages and 1 appendix.

Canadian Space Agency (2004), Memorandum of Understanding with the Communications Research Centre Canada Concerning the Management of the Cascade Technology Demonstration Project in the CASSIOPE Mission, and other CRC Support to CSA, June 25 2004, 8 pages.

Peruzzini (W.) and Brassard (G.) (2004), Canadian Multi-Mission Small and Micro Satellite Buses, 4 pages.

Canadian Space Agency (2004), Contract no 9F028-034406/001/SR with Bristol Aerospace Limited for the Preliminary Design of a Small Satellite Bus, February 20 2004, 12 pages and 3 appendices.

Canadian Space Agency (2004), Contribution Agreement no LS2003-002 with the Governors of the University of Calgary Concerning Development and Flight of an Enhanced - Polar Outflow Probe, February 04 2004, 10 pages and 6 appendices.

Canadian Space Agency (2003), Contribution Agreement no LS2003-001 with Cascade Data Services inc. and MacDonald Dettwiler and Associates Ltd Concerning the CASSIOPE Mission, December 08 2003, 33 pages and 2 appendices.

Canadian Space Agency (2003), Terms and Conditions - Establishment of a Contribution Program to Support the Joint Cascade Technology Demonstration/Enhanced-Polar Outflow Probe Small Satellite (CASSIOPE) Mission, October 30 2003, 24 pages and 5 appendices.

Canadian Space Agency (2003), Policy and Procedures for Evaluation, October29 2003, 14 pages.

Canadian Space Agency (2003), CASSIOPE Mission Management (Cascade, SmallSAT Bus, ePOP combined Mission) - Project Approval Document, September 10 2003, 14 pages.

Canadian Space Agency (2003), CASSIOPE Contribution Program - Submission to PRAB, September 10 2003, 4 pages.

Canadian Space Agency (2003), Due Diligence - MDA Cascade Evaluation, Report prepared by Mantrix inc., 25 pages.

Canadian Space Agency (2002), Satellite Communications Service Line Strategic Plan (2002/2003 to 2007/2008), 12 pages and 8 appendices.

Peruzzini (W.) and Brassard (G.) (2000), Small and Micro Satellite Program at the CSA, CASI Conference, 5 pages.

* * * * * * *

Macdonald Dettwiler and Associates Ltd (2007), CASSIOPE Quarterly Program Review (March 08 2007), 21 pages.

Macdonald Dettwiler and Associates Ltd (2007), CASSIOPE Monthly Status and Claim Report (February 21 2007), 30 pages and 1 appendix.

Macdonald Dettwiler and Associates Ltd (2007), Progress Report - CASSIOPE Bus Procurement (Period of January 2007), February 06 2007, 19 pages and 8 appendices.

Macdonald Dettwiler and Associates Ltd (2006), CASSIOPE Quarterly Program Review (December 07 2006), 17 pages.

Macdonald Dettwiler and Associates Ltd (2006), CASSIOPE Quarterly Program Review (September 12 2006), 25 pages.

Macdonald Dettwiler and Associates Ltd (2006), CASSIOPE Quarterly Program Review (May 03 2006), 20 pages.

Macdonald Dettwiler and Associates Ltd (2006), CASSIOPE Senior Management Meeting (April 12 2006), 25 pages.

Macdonald Dettwiler and Associates Ltd (2006), CASSIOPE Mission Critical Design Review (March 28, 29 and 30 2006), 238 pages.

Macdonald Dettwiler and Associates Ltd (2005), CASSIOPE Quarterly Program Review (July 07 2005), 21 pages.

Macdonald Dettwiler and Associates Ltd (2005), CASSIOPE Quarterly Program Review (April 19 2005), 19 pages.

Macdonald Dettwiler and Associates Ltd (2005), CASSIOPE Quarterly Program Review (February 03 2005), 20 pages.

Macdonald Dettwiler and Associates Ltd (2004), CASSIOPE Quarterly Program Review (October 06 2004), 21 pages.

Macdonald Dettwiler and Associates Ltd (2004), CASSIOPE Quarterly Program Review (July 12 2004), 16 pages.

Macdonald Dettwiler and Associates Ltd (2004), CASSIOPE Quarterly Program Review (April 19 2004), 19 pages.

Macdonald Dettwiler and Associates Ltd (2003), Program Kick-Off Meeting (December 15 2003), 64 pages.

* * * * * * *

University of Calgary (2007), CASSIOPE Quarterly Program Review - ePOP (March 08 2007), 14 pages and 32 pages.

University of Calgary (2007), CASSIOPE/ ePOP Progress Report - Submitted to the CSA Under the Contribution Agreement Between CSA and the University of Calgary for the Period of January 2007 (February 14 2007), 19 pages.

University of Calgary (2006), CASSIOPE Quarterly Program Review - ePOP (September 12 2006), 14 pages.

University of Calgary (2005), CASSIOPE Quarterly Program Review - ePOP (July 07 2005), 16 pages.

University of Calgary (2005), CASSIOPE Quarterly Program Review - ePOP (April 19 2005), 16 pages.

University of Calgary (2005), CASSIOPE Quarterly Program Review - ePOP (February 03 2005), 31 pages.

University of Calgary (2004), CASSIOPE Quarterly Program Review - ePOP (October 06 2004), 13 pages.

University of Calgary (2004), CASSIOPE Quarterly Program Review - ePOP (July 12 2004), 15 pages.

University of Calgary (2004), CASSIOPE Quarterly Program Review - ePOP (April 19 2004), 20 pages.

University of Calgary (2004), Agreement on the ePOP Payload in the CASSIOPE Mission Among the Governors of the University of Calgary, and Cascade Data Services inc., and MacDonald Dettwiler and Associates Ltd, April 14 2004, 15 pages and 1 appendix.

Appendix - Management Action Plan (Updated April 2009)

Ref. Recommendation Identified Responsibility Action Plan Details Timetable
Organization Function
1. That the Canadian Space Agency require MacDonald Dettwiler Associates Ltd/Cascade Data Services inc. to finalize by the end of December 2007 a new and completely updated Cascade business plan detailing the major scenarios for attracting potential investors and penetrating potential markets for the Cascade technology. Space Programs Branch Senior project manager Space Programs Branch received in September 2007 an updated Cascade business plan from MDA/CDS. The business plan is updated and presented every three months at the Quarterly Program Review. In December 2008, MDA/CDS reached an agreement with another firm (O3B) and proposed a revised business plan for the commercialization phase. Technology Partnerships Canada is reviewing the proposal. September 2007
2. That the Canadian Space Agency implement an ongoing monitoring and reporting mechanism, in coordination with Technology Partnerships Canada, on potential investors and potential markets for the Cascade technology. Space Technologies Branch Director DTMA In preparation to the transition from Space Programs Branch to Space Technologies Branch for management of the contribution agreement between the CSA and MDA/CDS after the successful commissioning of CASSIOPE, the manager, Satellite Communications Applications Spectrum will work closely with Technology Partnerships Canada and the CASSIOPE program management office to establish monitoring mechanisms of the Cascade business plan. Beginning of FY 2009-2010
3. That the Canadian Space Agency strengthen the program management office by adding on an ad hoc basis a specialized resource in financing and marketing in the space sector to proactively interact on an ongoing basis with the MacDonald Dettwiler and Associates Ltd/Cascade Data Services inc. business development team. Space Technologies Branch Director DTMA The manager, Satellite Communications Applications Spectrum will ensure that the necessary expertise to interact with MDA/CDS is available as soon as possible and will work closely with the CASSIOPE program management office and Technology Partnerships Canada. FY 2009-2010

Footnotes

Footnote 1

Canadian Space Agency (2007), CASSIOPE Contribution Program Poster as of February 28, 2007.

Return to footnote 1 referrer

Footnote 2

Canadian Space Agency (2003), CASSIOPE Contribution Program - Submission to PRAB, September 10 2003, page 1.

Return to footnote 2 referrer

Footnote 3

Canadian Space Agency (2003), CASSIOPE Mission Management (Cascade, SmallSAT Bus, ePOP combined Mission) - Project Approval Document, September 10 2003, page 3.

Return to footnote 3 referrer

Footnote 4

Canadian Space Agency (2005), Enhanced - Polar Outflow Probe - Project Approval Document, revised February 15 2005, pages 4-5.

Return to footnote 4 referrer

Footnote 5

Canadian Space Agency (2006), Report on Plans and Priorities. 2006-2007 Estimates, pages 14 and 18.

Return to footnote 5 referrer

Footnote 6

Canadian Space Agency (2003), Terms and Conditions - Establishment of a Contribution Program to Support the Joint Cascade Technology Demonstration/Enhanced-Polar Outflow Probe Small Satellite (CASSIOPE) Mission, October 30 2003, pages 38 and 47.

Return to footnote 6 referrer

Footnote 7

Canadian Space Agency (2003), CASSIOPE Mission Management (Cascade, SmallSAT Bus, ePOP combined Mission) - Project Approval Document, September 10 2003, page 10.

Return to footnote 7 referrer

Footnote 8

Canadian Space Agency (2003), Due Diligence - MDA Cascade Evaluation, Report prepared by Mantrix inc., page iii.

Return to footnote 8 referrer

Footnote 9

Canadian Space Agency (2003), Contribution Agreement no LS2003-001 with Cascade Data Services inc. and MacDonald Dettwiler and Associates Ltd Concerning the CASSIOPE Mission, December 08 2003, article 6.3.

Return to footnote 9 referrer