Management Action Plans Follow-up for evaluation - Annual Report as of March 31, 2012

Download the PDF version (404 KB)

Audit and Evaluation Directorate

April 2012

Implementation summary

This follow-up report on the implementation of management action plans outlines the measures taken by the various entities concerned in response to findings and recommendations. As part of the follow-up process, management action plans are reviewed annually until they are fully implemented, and the extent of implementation is assessed and reported to the Executive Committee, which serves as the Canadian Space Agency's (CSA) Departmental Evaluation Committee.

This annual report contains the follow-up findings, as of March 31st, 2012, of two evaluation projects for which reports and management action plans have been submitted to and approved by the Executive Committee. The following charts give an overview of the implementation status of the management action plan elements.

To be done (1 management action plan element) In progress ≤ 50% (3 management action plan elements) In progress > 50% (5 management action plan elements) Completed 100% (2 management action plan elements)
Management action plan elements 1 3 5 2

The following pages set out in detail the progress of the action plans for each of the evaluation projects.

Summative Evaluation of the 2000-2009 Canada / ESA Cooperation Agreement

Evaluation project: Goss Gilroy

Evaluation objective

This summative evaluation was conducted by Goss Gilroy Inc (GGI) Management Consultants in 2008 and was designed to assess relevance, success and cost-effectiveness of the Canada/ESA agreement. The evaluation report was presented to the Executive Committee in June 2010 and subsequently approved in September 2010.

Program support & communication Policy monitoring Program renewal Priority alignment
40% 20% 20% 20%

Nature of recommendations

The evaluation report provided recommendations relating to program renewal, strategic priority alignment, program support and communication to Canadian industry and finally, policy monitoring. Specifically, the five recommendations for program improvement were:

  1. Renew the Canada/ESA agreement;
  2. Clarify ESA's role within the Canadian Space Agency's (CSA) priorities;
  3. Develop and publicize policies to support organizations and new entrants;
  4. Develop a plan for communicating ESA industrial opportunities; and
  5. Monitor impacts of European Space Policy and European Union (EU) trends on Canada.

Implementation status

Recommendation 1, which relates to renewing the agreement for an additional 10 year period, is in progress and is more than 50% complete. The CSA has taken all the required steps to renew the Canada-ESA Agreement. The Agreement was signed in December 2010 and the treaty is currently in the ratification process. It is expected to take effect in 2012.

Recommendation 2 relates to clarifying ESA's role within the long-term plan. This recommendation has yet to be implemented due to circumstances beyond the control of ESA program management.

Recommendations 3 and 4 relate to project support and communication. These recommendations are in progress and are less than 50% complete. Although some efforts have been made to develop policies to support and inform potential Canadian bidders and to update the ESA management framework documents, such as the Guide for Canadian Delegates, a staffing shortage has delayed progress in this area.

Implementation Status
To do 0% 1
In progress ≤ 50% 2
In progress > 50% 2
Completed 100% 0

Finally, recommendation 5 relates to regular and continuous monitoring of the impacts the European Space Policy may have on Canada's participation in European space activities. Significant efforts have been made by the CSA's Policy and External Relations group to keep abreast of the policy developments by involving the Department of Foreign Affairs and International Trade (DFAIT) and the Canadian Mission in Brussels. Also, there is a new Executive Director of Partnership with EU who will be actively involved in this continuous policy monitoring and assessment. Although recommendation 5 is ongoing, it is assessed as in progress and more than 50% complete because monitoring the development of policies and the subsequent impacts they have on Canada takes time, however the mechanisms are in place at the CSA and DFAIT to ensure continuous monitoring. At the moment of this follow-up, the new Executive Director of Partnership with EU had yet to present an update on the developments and impacts of the European Space Policy on Canada to senior management. The actions associated with this recommendation will be re-assesed next year for completion when further evidence of continuous and regular reporting to CSA senior management has been carried out.

Summative Evaluation of the STDP for the Period of 2002/03 to 2007/08

Evaluation project: 570-2800

Evaluation objective

Government Consulting Services (GCS) conducted the evaluation of the Space Technology Development Program (STDP) (now referred to as Enabling Technology Development) in 2009-10 and the report was subsequently approved by the Executive Committee in June 2011. The evaluation focused on four main areas: program relevance, program design and delivery, program success, and cost-effectiveness.

Project support and communication Program objectives Project management
33% 34% 33%

Nature of recommendations

The evaluation report shed light on the need to employ the appropriate funding mechanism and identify R&D technology requirements in order to attain the program's objectives, to improve project management through more efficient procurement processes and service standards, and to provide project support and establish communication strategies with companies and CSA clients. The evaluation report contained six recommendations for program improvement.

  1. Clarify program objectives;
  2. Map research and development (R&D) technology requirements with all proposed missions;
  3. Clarify procurement processes between CSA and Public Works and Government Services Canada (PWGSC) to reduce delays;
  4. Consider the viability of a two-step application process;
  5. Provide support to companies for the preparation of proposals; and
  6. Implement a formal communication strategy to communicate project results to CSA clients.

Implementation status

Recommendations 1 and 2, which relate to program objectives, are in progress and are more than 50% complete. Program objectives have been clarified in light of CSA's re-organization and the new Program Activity Architecture (PAA). The Enabling Technology Development (ETD) group is currently developing a Performance Measurement Strategy with an updated logic model, which serves as the program's roadmap and includes both contracts and contributions as outputs to achieve expected outcomes. Contributions to industry for R&D funding may be introduced in 2013-2014, if the budget permits. A functional framework to use transfer payments has been partially established and a little less than half of the ETD staff have been trained in the use of G&Cs authorities. A standardized process through which R&D technology needs are identified and prioritized has been developed and is slated to be implemented throughout 2012-13.

Recommendations 3 and 4, which relate to program management, have been fully implemented. A CSA working group was formed to address CSA-wide PWGSC issues (such as delays around RFP and contract issuance). Two main tools were developed to clarify the major steps in the contract approval process, as well as the roles and responsibilities of each party involved. Although ETD considered the viability of introducing a two-step application process for G&C authorities, there are currently no available G&C funds for the next two years.

Implementation Status
To do 0% 0
In progress ≤ 50% 1
In progress > 50% 3
Completed 100% 2

Recommendations 5 and 6 relate to project support and communication. Although there have not been significant efforts made to prepare formalized documents or templates to assist companies unfamiliar with the process for responding to contracts, ETD is planning to address this topic and to present instructions to bidders at the upcoming Technology Forum in May 2012. Recommendation 5 is in progress, but less than 50% complete. Finally, with identification of a client for every technology priority, ETD has formalized the way it communicates project results to CSA clients by consulting and inviting the client to project kickoff meetings and providing the client with copies of the Final Review Packages. A client survey has also been developed and feedback from these surveys will be compiled to produce client satisfaction reports. Recommendation 6 is in progress and more than 50% complete.