Summative Evaluation of the Class Grant and Contribution Program

Download the PDF version (858 KB)

Prepared for: Canadian Space Agency

Project Number: 570-2745
SIGMA WBSE: C.000255.001
November 13, 2008

Table of Contents

1. Background and Context

This report presents the evaluation findings of the Canadian Space Agency's (CSA) Class Grant and Contribution (G&C) Program. The Class G&C Program to Support Awareness, Research and Training in Space Science and Technology was first approved in September 2002 for a period of five years ending on March 31, 2008. As the terms and conditions of the Program reach expiry, the CSA is required to undertake a summative evaluation of the Program in accordance with Article 7.3.7. of the Policy on Transfer Payments of the Treasury Board of Canada, Secretariat (TBS). The objective of the study was to evaluate the Program in terms of relevance, success and cost-effectiveness. The evaluation study was conducted by Government Consulting Services (GCS) on behalf of the CSA between July 2008 and November 2008.

1.1 Description

Prior to the creation of the Class G&C Program, the grant and contribution structure of the CSA was characterized by a few large programs and a significant number of small programs each assigned with their own budgets. It was the view of senior management, at the time, that this fragmented approach was hindering the Agency's capability to adjust budgets during the fiscal year to meet the needs of targeted clientele by moving funds from low to high demand initiatives.

As such, a decision was taken to merge several small programs with similar objectives under a single budget. It was anticipated that this approach would help streamline the administration of grants and contributions while providing the Programmatic capability and the financial flexibility required to support space-related research, development and demonstration activities in collaboration with CSA's partners.

The Class Grant and Contribution Program was established through a decision of the Treasury Board dated August 21, 2002, which authorized the creation of ten program components. Subsequent decisions of the Treasury Board dated February 15, 2005 and January 31, 2007, established two additional components while modifying the terms and conditions of certain existing components. The components are referred to as "Annexes" for consistency with the Treasury Board submissions from which they were established. See Appendix A for a description of each Annex.

At its broadest level, the primary objective of the Class G&C Program is to build the capacity of the Canadian space sector in space science disciplines and space technology areas of priority to the Canadian Space Program.

The primary objective of the Class G&C Program is supported by three sub-objectives that are:

  1. To increase the awareness of young Canadians regarding science and technology;
  2. To build the human resources capacity of the Canadian space sector; and
  3. To build the research capacity of the Canadian space sector.

The Program strives to achieve these objectives by issuing grants and contributions to the CSA's partners in the Canadian space sector, which is consistent with the functions of the Agency specified in Article 5.3 (c) of the Canadian Space Agency Act.

All components of the Program are designed to involve collaboration with a wide range of partners such as other government departments, universities, academics, not-for-profit research organizations, international non-government organizations and Canadian industry. However in each case, the CSA bears unique responsibility for program delivery and administration.

1.2 Resources

The 2002 TB decision authorized the CSA to issue grants of up to $3 million per year and contributions of up to $2 million per year for the period from 2003-2004 to 2007-2008. The 2007 TB decision increased the Program's overall annual authority limit for the issuance of grants by up to $1 million for a revised annual authority limit of up to $4 million in 2007-2008.

The Program is chargeable to CSA Vote 40 – Grants and Contributions and appears in the Main Estimates as a single line item entitled "Grants and Contributions Program to Support Awareness, Research and Training in Space Science and Technology."

The costs incurred by CSA in managing and administering the Class G&C Program are difficult to ascertain as they are funded from existing reference levels and have not been directly allocated to the Program.

1.3 Governance

Program management and delivery is dispersed among five different entities at CSA including Space Technologies Branch, Space Science Branch, Canadian Astronaut Office (CAO), Communications Directorate and External Relations Directorate. For some Annexes, there is joint accountability for management and results whereas other Annexes were accorded to specific entities. Each Branch and Division is also responsible for the day to day administration of separate Annexes. See Appendix B for a summary table of the governance structure.

2. Evaluation Methodology

The study was conducted based on the Evaluation Plan developed by GCS in August 2008. As part of the planning for the evaluation, the original logic models were updated to clearly reflect the objectives of the Program as presented in the 2002 TB submission and subsequent updates in 2005 and 2007. The revised logic models categorize each Annex within three main components: Awareness, Human Resources and Research. The components represent an alignment of Annexes based on common objectives and expected results at the immediate and intermediate levels. Revised logic models for the awareness, human resources and research components of the Class G&C Program can be found in Appendix C.

2.1 Evaluation Issues and Questions

The evaluation was designed to address three key issues: relevance, success and cost-effectiveness/alternatives. The issue area of success was further examined by component area as described above. A total of 16 evaluation questions were explored, as depicted in Table 1. The complete evaluation matrix, including indicators, related data sources and collection methods is presented in Appendix D. Appendix E provides a crosswalk between the evaluation matrix and the policy tests of the Expenditure Review Committee's strategic review guidance for 08/09.

Table 1: Summary of Evaluation Questions

Evaluation Issue Evaluation Question
Relevance
  1. Is the Program aligned with the mandate of the Canadian Space Agency, the Canadian Space Strategy and Government of Canada priorities?
  2. Does the Program continue to address an ongoing need for which it was originally established?
Success – Awareness Component
  1. Is the correct group being targeted by the awareness component of the Program and is the method of intervention appropriate to their needs?
  2. To what extent does the awareness component of the Program reach young Canadians?
  3. Do young Canadians have a greater awareness of S&T as a result of the awareness component of the Program?
Success – Human Resources Component
  1. Are recipients acquiring relevant expertise from the HR component of the Program?
  2. Are recipients transmitting their expertise to CSA partners who were not involved in the learning project?
  3. Are recipients gaining employment in the Canadian space sector or other scientific disciplines, either domestically or abroad?
Success – Research Component
  1. Are recipients generating useful research results from the research component of the Program?
  2. Are recipients transmitting research results to CSA partners who were not involved in the research project?
  3. Are research results being applied to the Canadian Space Program or other scientific disciplines, either domestically or abroad?
Success – Other
  1. Has the Program fostered increased collaboration between stakeholders in the space sector that has helped them to better achieve mutual objectives?
  2. Has the Program resulted in any unintended impacts (positive or negative)?
  3. Have issues addressed in previous audits, reviews and evaluations been addressed?
Cost-Effectiveness / Alternatives
  1. Would recipients have achieved the same results from their projects without financial assistance from the Program?
  2. Are there more cost-effective ways to achieve the same outcomes as the Program?

2.2 Data Collection Methodologies

The following methodologies were used as sources of information for the summative evaluation:

  1. Document review;
  2. Key informant interviews (managers and recipients);
  3. Comparative analysis; and
  4. Focus groups.

These methods are described in more detail in the following sections.

Document Review

A review of relevant documentation was used to inform the development of instruments for the key informant interviews as well as one of the lines of evidence for the evaluation. Documents reviewed for this evaluation are listed in Appendix F. These included, but were not limited to:

  • Background and contextual documents such as CSA's Management Accountability Framework (MAF) Assessment and the National Aerospace and Defence Strategic Framework;
  • Inception and planning documents that detail the intended objectives and outcomes of the Program (e.g., Treasury Board Submission, Logic Model); and
  • Operational documents and data such as the Minutes and Records of Decision from Senior Space Science Advisory Committee, the Mid-Term Review of the Class Grant and Contribution Programs, survey and annual reports by specific Annex.

A document review template was used to capture relevant information from the documents according to the indicators in the evaluation matrix.

Comparative Analysis

A comparative analysis of similar G&C programs within the Government of Canada was conducted with the goal of identifying best practices for cost-effective program delivery. The following four programs were reviewed:

  • Research Partnership Program, Natural Sciences and Engineering Research Council;
  • Agricultural Bioproducts Innovation Program, Agriculture and Agri-food Canada;
  • Youth Employment Strategy (Youth Awareness Initiative), Human Resources and Social Development Canada; and
  • Going Global Science and Technology Program, Department of Foreign Affairs and International Trade.

The comparative analysis did not yield the results anticipated for identifying best practices for cost-effective program delivery, however, the research was used to inform and provide insight into the interpretation and analysis of evaluation findings.

Key Informant Interviews

Key informant interviews served as an important source of information for the evaluation, by providing information on stakeholder views in response to all of the evaluation questions. A total of 37 interviews were conducted with stakeholders. A list of interviewees is provided in Appendix G. Appendix H includes the guides used to conduct the interviews. Table 2 outlines the categories of individuals that were interviewed as well as the number of interviews conducted by the project team.

Table 2: Number of Interviews Conducted, by Interview Group

Stakeholder Group Number of Interviews
CSA Senior Management 4
CSA Program Management 8
Recipients of G&C Program Funding 25
Total 37

A proportional distribution of recipients of Class G&C Program funding from each of the 12 Annexes was sought to participate in interviews. No recipient interviews were conducted for Annexes V and XI as there have not been any projects funded since inception of the Program. Table 3 shows the number of recipient interviews conducted by Annex.

Table 3: Number of Recipient Interviews Conducted, by Annex

Annex Number of Recipient Interviews
Annex I 2
Annex II 3
Annex III 3
Annex IV 2
Annex VFootnote 1 -
Annex VI 2
Annex VII 6
Annex VIII 3
Annex IX 2
Annex X -
Annex XIFootnote 1 -
Annex XII 2
Total 25

Focus Groups

Two focus groups were conducted with key stakeholders in the Canadian space sector. Participants included industry, academia, not-for-profit research organizations and international non-government organizations as well as members of CSA's scientific advisory committees. The goal of the focus groups was to assess the perceptions of key stakeholders in the Canadian space sector with regard to the rationale, impact and cost-effectiveness of the Class G&C Program. See Appendix I for a list or focus group participants.

2.3 Constraints and Limitations of the Methodology

As with most evaluations, there were constraints and limitations with the methodology used for the evaluation. They were as follows:

Sample Size for Recipient Interviews

Although the total number of recipient interviews conducted for the evaluation is reasonable, the resulting number per Annex was quite small. Budget and timeframes for the evaluation required that interviews be limited. Therefore, any evidence gathered from interviews by Annex cannot be considered representative of the entire population of stakeholders.

Qualitative-Based Evidence

The evidence for this evaluation is largely qualitative, due in part, to the challenges in quantifying the results of some of the projects being funded by the Program. The vast majority of performance information that has been collected by the CSA pertains to the outputs of various Annexes and is, by nature, volumetric information. Also, due to the small sample size of interviews by Annex, it was not possible to quantify the results, therefore this evidence remained anecdotal.

Timeframe of Results Achievement

In the short term, the extent to which advanced research in space science and technology areas has been conducted is easily measured. However, the extent to which these research results are being applied to the Canadian Space Program or other scientific disciplines will only materialize in the longer term, beyond the five year lifespan of the Program. For the purposes of this evaluation, it was reasonable to not expect any evidence in this regard.

Attribution of Results

In consideration of the many, wide ranging factors that conspire to increase the HR and research capacity of the Canadian space sector, it is clear that the Class G&C Program is but a small contributor. As such, the Project Team has identified from the outset that it is not reasonable to measure or attribute a significant weight to the Program's contribution towards the attainment of ultimate outcomes.

Small Scope

A final limitation of the methodology is related to budget and timing. Since the Class G&C Program is relatively small ($5 million in annual G&C funding), it would not be financially prudent to expend significant time and resources on a large-scale evaluation. As such, the evaluation approach has been tailored to reliably answer key questions required of a summative evaluation with an economical budget and within a reasonable timeframe.

3. Key Findings

This section of the report presents a summary of the evaluation findings which are organized into the issue areas of relevance, success and cost-effectiveness and are presented by evaluation question, as found in the evaluation matrix.

3.1 Relevance

The findings of the evaluation on the issue of relevance are presented in this section of the report. The evaluation questions that were considered in addressing this issue were as follows:

  • Is the Program aligned with the mandate of the Canadian Space Agency, the Canadian Space Strategy and Government of Canada priorities?
  • Does the Program continue to address an ongoing need for which it was originally established?

3.1.1 Program Alignment

Conclusion

The Class G&C Program is aligned with Government of Canada priorities, with the stated objectives of the Canadian Space Strategy and with the mandate of the CSA. The Program reflects these priorities by focusing on the benefits of a vibrant space sector through outreach, research and the development of globally connected and highly qualified human resources.

The CSA Mandate is to promote the peaceful use and development of space, to advance the knowledge of space through science and to ensure that space science and technology provide social and economic benefits for Canadians. The Class G&C Program aligns with the CSA Mandate with respect to advancing the knowledge of space through sciences.

The formal objective of the Program is to build the capacity of the Canadian space sector in space science disciplines and space technology areas of priority to the Canadian Space Program. This primary objective of the Class G&C Program is supported by three sub-objectives that are:

  1. To increase the awareness of young Canadians regarding science and technology;
  2. To build the human resources capacity of the Canadian space sector; and
  3. To build the research capacity of the Canadian space sector.

These objectives clearly continue to support the Canadian Space StrategyFootnote 2 in particular in its priority for Space Awareness and Learning. They are also clearly aligned with the Government of Canada's science and technology strategyFootnote 3 which has been designed to ensure a benefit to Canadians from science and technology, most notably through the development of highly skilled human resources. Specifically, the Canadian Space Strategy focuses on the following aspects:

  • Entrepreneurial Advantage;
  • Knowledge Advantage: Focussing Strategically on Research in the National Interest; and
  • People Advantage:
    • Enhancing Opportunities for Science and Technology Graduates;
    • Increasing the Supply of Highly Qualified and Globally Connected Science and Technology Graduates; and
    • Getting Canadians Excited about Science and Technology.

Management and recipient interviews indicated that the Class G&C Program is designed to foster research which will benefit the space sector and is focused on developing highly qualified personnel (HQPs) who have international space sector connections. The Program also focuses on leverage and networks which is well aligned with the CSA's pledge to cooperate and work with partners for mutual benefit.

Taken together, all focus on the practical use of science and technology to support the well being of Canadians, through research and the development of trained personnel. The ultimate goal is to produce highly trained personnel who will drive innovation for the economic and social good of all Canadians.

3.1.2 Program Alignment

Conclusion

There is a continued need for the Class G&C Program to ensure innovative capacity and human resources capacity in the Canadian space sector. The CSA is the only organization that is poised to offer focused support to stakeholders in achieving these objectives. The Class G&C Program provides the financial support in areas in which other sources of funding do not exist.

Innovative Capacity

Interviews confirmed the importance of creating innovative capacity to support international space ventures and to support Canadian research expertise. For example, while the ESA and Japanese space agency have both developed robotic arms, the Canadarm's first mover advantage has ensured future projects for Canada such as the current Dextre module on the International Space Station. Innovative capacity does translate into future success.

Interviews suggested that because of the small size of the Canadian space sector, the critical mass to achieve broadly based innovative capacity is difficult to attain within academia and industry without the explicit support and assistance of the government. Both funds and expertise are in short supply.

The CSA plays a role in addressing this need by coordinating between different actors and through funding. The Class G&C Program is instrumental in both coordinating activities and helping to leverage funds from other sources. As such, the CSA is not fully funding research, but helping to draw funds from other areas such as industry and academia. Much of the research being undertaken has practical use in space missions in which Canada does or will participate. Interviewees and focus group participants were unanimous in that if the CSA did not take the lead, it would be very difficult to develop innovative capacity.

Human Resources Capacity

The space sector places a great deal of emphasis on HQPs with unique skill sets applicable to space sector issues. Both the CSA and industry have shown interest in individuals with a space sector background for employment.

Interviews with CSA management and industry indicated that while there is a need for individuals who both understand the space sector and have relevant skills to support the Canadian space strategy, industry and academia alone are unable to develop such skills. Canada produces many engineers and scientists, however, their skill set does not focus on the unique problems inherent in space science. Without some form of intervention, there is a risk that few professionals would be developed and available for employment in the CSA, academia or industry due to the small size of the Canadian space sector.

3.2 Success – Awareness Component

The findings of the evaluation on the issue of success for the awareness component are presented in this section of the report. The awareness component is comprised of the following two Annexes:

  • Annex IX – Youth Space Awareness Contributions Program; and
  • Annex XII – Space Awareness and Learning Grant Program.

The evaluation questions that were considered in addressing this issue were as follows:

  • Is the correct group being targeted by the awareness component of the Program and is the method of intervention appropriate to their needs?
  • To what extent does the awareness component of the Program reach young Canadians?
  • Do young Canadians have a greater awareness of Science and Technology (S&T) as a result of the awareness component of the Program?

3.2.1 Alignment with Needs of Target Groups

Conclusion

The groups being directly targeted by the awareness component of the Program are learning organization animators, educators and Canadian students at the primary, secondary and post-secondary levels. Educators and learning organization animators are the intermediaries in promoting and enhancing the interest and awareness of Canadian youth with limited knowledge of space activities and equally limited opportunities to learn about science and technology in the Canadian space context. The evaluation found that these target groups are the most appropriate beneficiaries for the Program.

The methods of delivery, although different between the grant and contribution annexes, are focused on the right types of intervention. University students have been given opportunities to attend conferences and workshops they otherwise would not be able to attend. Primary and secondary students have been able to participate in space and science-focused competitions and events otherwise not accessible to this group due to funding issues. Funding for educational materials, has allowed both the curricular and extracurricular learning communities to produce and incorporate material within their exhibits and youth-focused learning programs with the aim of raising awareness of space science and technology topics among Canadian youth.

According to management interviews, the outreach program has the dual goals of both raising scientific literacy in general and in steering individuals towards careers in the space field. The Government of Canada's science and technology strategy supports the development of scientific awareness and capacity in Canada as a means to fostering future innovation.

Young Canadians have been direct beneficiaries of the Grants Program through their participation in conferences, workshops and space and science-focused competitions. They have also been indirect beneficiaries of the Contributions Program, through their exposure to Canadian Space content, materials, learning programs and exhibits developed by the curricular and extra-curricular learning communities.

The first target group for the awareness component is comprised of learning organization animators, employed by educational non-profit organizations, such as planetariums, space camps and science centres and educators. This group offers science and technology-focused learning programs, materials, events and exhibits designed for youth, who are often elementary and middle school students. Interviewees indicated that research demonstrates the benefits of engaging youth in science and technology at an early age in order to encourage interest in the sector. By exposing youth to possibilities in scientific and engineering fields, they are able to foster future innovation and technology as drivers of economic growth.

Space-focused learning programs, materials, events and exhibits development require funding, however, and the limited budgets of these organizations do not permit extensive projects. The Class G&C Program allows for the development of educational content with a Canadian space focus, permitting projects that would otherwise not be feasible.

Students at the elementary, secondary and university levels make up the second target group. These students participate in the Program by attending conferences, workshops or events which have a Canadian space-focused component. Many interviewees indicated that there is an importance in educating people just as they are to embark on their careers paths and inform them of opportunities in the space sector. The events these students attend may open their eyes to opportunities or inspire them to pursue a particular career direction.

However, the programs are not free and it would be difficult for students to attend without sponsorship. The Grants element of the Program fills this gap by funding travel and participation that would otherwise not occur.

3.2.2 Extent of Reach

Conclusion

The Class G&C Program appears to have reached a large number of individuals including elementary, secondary and post-secondary students, learning organization animators and educators. In terms of volume, Program documentation indicates that forty-six projects have been awarded funding to date and one hundred and sixty grants have been awarded to individuals. However, the total number of youths reached by the Program cannot be ascertained from the available data.

Forty-six projects have been awarded funding to dateFootnote 4 which have been viewed by a large number of youth. For example, one recipient interviewee indicated that their program had engaged over thirty one thousand youth in one hundred and fifty communities across Canada. If each program has comparable statistics, it would appear that G&C funding has a large scale reach among Canadian youth.

Funding went to such institutions as a planetarium in Montreal and a Science Centre in northern Ontario. Surveys sent to these institutions indicated that a large number of students were exposed to space topics, as a direct result of the Class G&C Program.

For elementary, secondary and university students, one hundred and sixty grants have been issued to date, with the majority awarded at the post-secondary level. These grants have been provided to students who have some curiosity about the space field and would like to benefit from further exposure. As such, they hold the potential to become future HQPs and require nurturing to pursue a career in space.

3.2.3 Awareness of Science and Technology

Conclusion

Anecdotal evidence suggests that participants are gaining a greater awareness of S&T through their exposure to topics of interest in the Canadian space sector. In applying for and receiving grants, students are required to identify specific learning objectives and subsequently to provide specific examples of learned space-related content as a result of their participation in space-focused conferences, workshops and competitions. However, the percentage of youths gaining increased awareness of S&T as a result of their participation in the Program cannot be determined from the available data.

Recipient interviews indicated that university students who received grants reported that attendance raised their awareness of many space-related topics in a relatively short time and in an effective manner and provided useful information and contacts which aided in planning their career goals. Management interviews suggested that many grant recipients have gone on to apply for the astronaut program, indicating an increased awareness of the space sector.

As part of the application process for the grants program, students are required to identify specific learning objectives related to the learning event that they plan to attend. Subsequent to the learning event and as a further condition of the grant, students are required to provide a report to the CSA demonstrating specific examples of learned space-related content as a result of their participation in space-focused conferences, workshops and competitions.

3.3 Success – Human Resources Component

The findings of the evaluation on the issue of success for the human resources component are presented in this section of the report. The human resources component is comprised of the following six Annexes:

  • Annex ICSA Supplements to NSERC Post-Graduate Scholarships and to Post-Doctoral Fellowships;
  • Annex IICSA Fellowships in Space Science, Space Technology and Aerospace Medicine;
  • Annex IVCSA Support to NSERC Industrial Research Chairs;
  • Annex VI – Aerospace Medicine Elective Award;
  • Annex VIIICSA Support to Scientific and Technical Space Conferences; and
  • Annex XCSA Support to Institutions Dedicated to Space Research and Education.

The evaluation questions that were considered in addressing this issue were as follows:

  • Are recipients acquiring relevant expertise from the HR component of the Program?
  • Are recipients transmitting their expertise to CSA partners who were not involved in the learning project?
  • Are recipients gaining employment in the Canadian space sector or other scientific disciplines in Canada?

3.3.1 Acquired Expertise

Conclusion

The Class G&C Program is providing funding to individuals who are acquiring valuable expertise in the space sector. While the many Annexes support human resource development in different ways, all have the stated objective of developing HQPs in the space sector. Participants have gained expertise from educational and research opportunities with a space-related focus. The selection process for grant recipients ensures funding is awarded only to those recipients who have demonstrated their focus on topics that are relevant to the CSA.

Alignment with Government of Canada (GoC) Priorities

The Government of Canada's Space Strategy has a stated focus on increasing the supply of highly qualified and globally connected science and technology graduatesFootnote 5. Annex I, II, IV, VI, VIII and X have the goal of increasing the supply of HQPs. Participants are only chosen on the basis of whether they will have a demonstrated focus on topics that are relevant to the space sector. Table 4 indicates the method by which each Annex increases the supply of HQPs.

Table 4: Method of Increasing HQPs Supply, by Annex

Annex Method of Increasing the Supply of HQPs
Annex I
  • Provides a supplement to students who will have an academic focus dealing with space-related topics
Annex II
  • Allows researchers to undertake research with a space focus while exposing their students to topics with a space topics
Annex IV
  • Provides funding to allow researchers to focus on space research
Annex VI
  • Allows participants to attend a four week space medicine training program and undertake space medicine-related research
Annex VIII
  • Space-related conferences support the diffusion of information with the goal of raising technical knowledge of the Canadian space community
Annex X
  • Provides scholarships for students to attend the International Space University (ISU) as well as for the operation of the Stratospheric Processes and their Role in Climate (SPARC) office

Annexes VI and X not only develop HQPs but are also able to produce globally connected S&T graduates. Participants are sponsored to attend research institutions where they will interact with participants from other space programs, building informal connections and knowledge which can later be used to support space activities within Canada. ISU students are able to not only meet individuals from other agencies, but are also given exposure to the activities of other space agencies.

Project Results Achievement

In the short-term, the expected results of Annexes in the Human Resources Component are to increase the expertise of human resources in space science and space technology areas. Table 5 indicates the number of recipients as well as their stage of completion of their respective programs. While not overly large, it should be noted that recipients are directly involved in space research and that the numbers involve individuals who are focused on the space sector.

Table 5: Short-Term Results, by Annex

Annex Number and Type of RecipientsFootnote 6
Annex I
  • 32 have received a grant: approximately 12 of 32 recipients have completed their postgraduate work and others are still continuing their studies
Annex II
  • 7 have received a grant: 3 recipients have found jobs in the space sector (national and international), 3 recipients have been involved in space missions
-
Annex IV
  • 4 recipients have involved 11 Post-doctoral fellows, graduate and undergraduate students have been involved in projects
Annex VI
  • 4 participants have been sponsored each year to participate at either Johnson Space Centre or Kennedy Space Centre for a total of 27
Annex X
  • 14 Canadian graduate students and young professionals have been associated with the Program
  • 16 Canadians have taken up careers relating to SPARC objectives during the period of the award
  • 57 students have been funded to attend the International Space University (ISU)

3.3.2 Knowledge Transfer

Conclusion

Participants within each Annex of the human resources component have been active in transferring their knowledge to other stakeholders. Supporting quantitative evidence indicates that several avenues have been used such as publishing papers, presenting at conferences and conducting workshops.

While the immediate focus of these Annexes is on the development of HQPs, knowledge transfer to other stakeholders does occur through papers and conference presentations. The method of knowledge transfer differs considerably by Annex, as illustrated in Table 6.

Table 6: Method of Transferring Knowledge, by Annex

Annex Method and Volume of Knowledge Transfer
Annex I
  • Interview results were not conclusive for this Annex as it is a supplement to graduate students to pursue their studies
Annex II
  • 5 papers have been published
  • 12 presentations have been made to conferences
Annex IV
  • Recipients have given 107 presentations and published or presented 87 papers in journals and at conferences
Annex VI
  • Recipients have been involved in both publishing papers and in making presentations to disseminate knowledge gained from their research and training
  • Often times the knowledge acquired has applicable results to the terrestrial environment and is both easily transferred and useful
Annex VIII
  • 857 presentations have been made at conferences
  • 1522 participants at the conference/workshop supported by the Program
  • 670 presentations at the conference/workshop were relevant to projects or programs supported by the CSA
Annex X
  • 140 publications in journals and presentations at conferences prepared by members supported by SPARC (institution) projects

3.3.3 Employment

Conclusion

Many grant recipients have found employment in the Canadian space sector. Due to the differing emphasis of each Annex, the success rate for recipients in gaining employment varies considerably across each Annex.

While the Annexes focus on improving human capital, not all recipients will necessarily work in the Canadian space sector. Annex VI recipients, for example, will generally work as medical doctors who can contribute their expertise to the CSA as well as apply space medicine knowledge to terrestrial medicine.

Nevertheless, many individuals have acquired skills deemed valuable by the space sector. Management interviewees noted that ISU graduates have invaluable knowledge that is not obtainable elsewhere and are great assets to the organizations they join.

Recipients from Annex IV indicated that while grants were helpful in gaining employment over the medium-term, these opportunities did not prove to be sustainable over the long-term. Once the funding of the industrial chair had been exhausted, the universities did not create permanent positions for the recipients as had been originally anticipated. Table 7 provides a summary of key employment indicators for the human resources component of the Program.

Table 7: Employment Indicators, by Annex

Annex Employment Indicators
Annex I
  • Many recipients have completed their studies and moved on to employment
  • The majority (but not all) have found employment in the Canadian space sector
Annex II
  • All four recipients indicated they had not found employment in a space-related field or in the institution where they had conducted their research
Annex X
  • 16 Canadians have taken up careers relating to SPARC objectives during the period of the award
  • Interviews indicated that ISU graduates possess skills and experience desirable to employers in the space field

3.4 Success – Research Component

The findings of the evaluation on the issue of success for the research component are presented in this section of the report. The research component is comprised of the following four Annexes:

  • Annex III - CSA Support to NSERC Partnerships Programs;
  • Annex V - Support to Not-For-Profit R&D Organizations for Space Research and Technology Development;
  • Annex VII - Space Science Enhancement Program; and
  • Annex XI - Joint CSA / Network Centres of Excellence Research Program.

The evaluation questions that were considered in addressing this issue were as follows:

  • Are recipients generating useful research results from the research component of the Program?
  • Are recipients transmitting research results to CSA partners who were not involved in the research project?
  • Are research results being applied to the Canadian Space Program or other scientific disciplines, either domestically or abroad?

3.4.1 Usefulness of Research Results

Conclusion

The research component of the Program is generating research results that are very useful for the Canadian space sector as evidenced by a high degree of alignment between individual projects and the priorities of the CSA. This conclusion is supported by evidence gathered from management interviews, recipient interviews, focus groups and the document review.

Project Objectives Achievement

The results of interviews and the document review generally indicated that projects achieved their research objectives. It is clear that the CSA is focusing on feasible and useful research that will have practical benefits in the future.

Alignment of Research Results with CSA and GoC Priorities

Interviews with program managers indicated that projects are only selected if they align with CSA and GoC priorities, which was confirmed by a review of funded project titles. Recipients indicated that funded research helps build Canadian technological capacity which can be utilized internationally, creating new opportunities for the CSA and its partners.

Focus groups also indicated that while previous initiatives focused only on providing technological solutions, current priorities demand the creation of a knowledge base through practical research. The Class G&C Program fulfills this priority by not only providing research but also by developing HQPs.

While funded research is directly applicable to the Canadian Space Program, much of if can be applied to terrestrial projects as well (for example, robotics). This facet supports GoC priorities for the space program in that there is a direct economic benefit.

3.4.2 Dissemination of Research Results

Conclusion

The research results generated by the research component of the Program are being shared by the stakeholders immediately involved in research projects. A broader dissemination of research results into the Canadian space sector has not been achieved due to the relatively short history of the Program in relation to the time required to publish peer reviewed papers and present them to the academic community.

Awareness of Research Results

Research results are transmitted either through academic means or through direct collaboration. The method of dissemination depends upon the Annex involved. Academic research can be disseminated through papers and conferences. However, research often takes years to produce results while publishing can take even more time. Since the Program is relatively young, the relative dearth of published data is an indicator of the early stage of the Program and not necessarily that results are not being disseminated.

Annex III projects by definition require cooperation and dissemination among the CSA, industry and academia. Research partnerships focus on projects which are deemed useful to the CSA but are too risky to be funded by industry. Research results are shared between partners and ensure dissemination across the space sector.

3.4.3 Application of Research Results

Conclusion

Much of the research generated by the research component of the Program is not yet being applied to the Canadian Space Program or other scientific disciplines due to the relatively short history of the Program. Given the long lead times required to plan and launch space missions, it is not reasonable to expect that these results would yet be achieved. However, there is some evidence that research at an advanced stage of completion is being utilized within the Canadian space sector.

Recipients mentioned that the collaborative aspect of the Class G&C Program encouraged them to focus their research on applicable uses, ensuring that funded research was not merely theoretical but practical. This was viewed as a positive aspect by recipients, as their research could then be utilized by other groups interested in related applications.

Management and recipients both indicated that many of the research projects are building a knowledge capacity which will be applicable in the near future, but that can not be immediately applied. This knowledge capacity ensures that Canadian capacity will be available when these opportunities eventually arise.

3.5 Success – Other

The findings of the evaluation on general issues of success are presented in this section of the report. The evaluation questions that were considered in addressing this issue were as follows:

  • Has the Program fostered increased collaboration between stakeholders in the space sector that has helped them to better achieve mutual objectives?
  • Has the Program resulted in any unintended impacts (positive or negative)?
  • Have issues addressed in previous audits, reviews and evaluations been addressed?

3.5.1 Benefits of Collaboration

Conclusion

Increased collaboration between stakeholders is a cornerstone of success for the Canadian space sector which is strongly favoured by all components of the Program. The interactions fostered by the Program are highly synergistic in nature, leading to the advancement of research projects, the creation of scientific knowledge and the exchange of information between participants.

Collaboration has allowed for significant leveraging of scarce funds and knowledge. Whereas an organization such as NASA has a large enough budget to independently realize projects, the Canadian space sector is too small to independently undertake many initiatives in isolation.

Cooperation between the CSA, industry and academia permits the space sector to share knowledge and leverage funds to further space sector objectives in increasing awareness, developing human resources and undertaking research. Interviews across all components consistently brought up the usefulness of collaboration in achieving mutual objectives. Collaboration also assisted in the dissemination of ideas across the sector and helped establish networks which would prove useful in projects other than those funded by the Class G&C Program.

The findings of interviewees and focus groups indicated that the Class G&C Program helped foster an enormous amount of collaboration. Due to the small size and limited funds of the Canadian space sector, collaboration and networking are extremely important in order to realize CSA goals. As such, the CSA plays a leadership role in the process that is beneficial and that has an impact much greater than the amount of money spent. The CSA's role as an enabler has been critical to the success of numerous projects.

3.5.2 Unintended Impacts

Conclusion

The primary unintended impact of the Program relates to an increased awareness and understanding of the importance of formal and informal networking by participants. Although participants were generally aware of the benefits of collaboration, it was not until they were compelled to do so by the Program that they began to realize significant benefits in this regard. However, respondents also noted that additional time and effort from management is required in order to ensure an adequate level of cooperation between stakeholders.

The ability to leverage CSA contributions both in resourcing and in participation was important in terms of getting other partners on board. Recipients were able to use CSA funding as leverage to attract more funding and to ensure greater collaboration with others.

Also, the application for funding provided a forum for feedback which would help focus researchers on practical results. One recipient noted that while they were rejected on their first application for funding, the CSA explanation allowed them to focus their proposal, subsequently gain $75,000, and leverage the money into other sources of funding for a total of $750,000. In this instance, the CSA provided focus which helped enormously in leveraging further funds.

A number of interviewees and focus group participants noted that the increased collaboration carried with it a cost in terms of time and money and that the objectives and agendas of all participants need to be considered. While collaboration is a good method of leveraging scarce resources, it does require additional time and effort to manage in order to ensure an adequate level of cooperation.

3.5.3 Previous Recommendations Implemented

Conclusion

CSA has addressed or is the process of addressing all issues raised in a mid-term review of the Program through a renewal process that will include new Terms and Conditions and a revised RMAF. However, the Program's current governance structure and performance measurement framework are inadequate to clearly demonstrate accountability for and achievement of results.

Mid-Term Review

The recommendations made to the Canadian Space Agency in its most recent mid-term reviewFootnote 7 were as follows:

  1. Ensure a revision of each component's objectives defined in their respective RMAF in order to specify the objectives further, to consolidate similar objectives, and consequently, to improve the Program's performance reporting;
  2. Reassess the level of funding in correlation to the defined objectives and expected results to answer Canadians' needs;
  3. Review and reinforce the Program governance structure;
  4. Ensure that the appropriate level of resources is assigned in order to manage the Program adequately;
  5. Explore further development of any reporting and monitoring tools for results, including the satisfaction level towards its service delivery, within the Program;
  6. Revise the data collection tools of the Program to ensure that they clearly incorporate the performance indicators used in their respective RMAF; and
  7. Ensure that Program tracking mechanisms of long-term outcomes will aggregately illustrate, within a logical manner, the teaching of components' objectives.

The seven issues identified by ASC have all been addressed or are in the process of being addressed by CSA management. The management response to the mid-term review provides a timetable for addressing the issues and recommendations identified by the report. The foundations for a more coordinated approach to program delivery have been laid with the identification of a champion and dedicated personnel for the administration of the Class G&C Program.

However, governance and performance measurement remain as issues to be addressed in the revised funding application. Although governance has improved over time, as was noted in the mid-term review, the current governance structure is not well defined in the face of program requirements.

Issues around performance measurement are not resolved for all Annexes but progress has been made towards a more credible and viable methodology. Volumetric indicators exist for many Annexes, which generally demonstrate that the required outputs are being produced. However, the majority of Annexes are not fully demonstrating the achievement of expected results through the use of direct quantitative indicators or client satisfaction survey results. Program managers have indicated the intent to prepare a Performance Measurement Strategy for the revised program, drawing upon the findings of this evaluation.

3.6 Cost-Effectiveness

The findings of the evaluation on the issue of cost-effectiveness are presented in this section of the report. The evaluation questions that were considered in addressing this issue were as follows:

  • Would recipients have achieved the same results from their projects without financial assistance from the Program?
  • Are there more cost-effective ways to achieve the same outcomes as the Program?

3.6.1 Achievement of Results without Assistance

Conclusion

The majority of participants would not have achieved the same results from their projects without financial assistance from the Program. Although CSA funding has not been as generous as desired, it has been leveraged by participants to attract additional funding from other sources. Funding was deemed unnecessary for Annex I as it was viewed as a bonus and did not draw participants to the space sector.

Management interviews and focus group responses were clear in that it would have been impossible for the CSA to achieve the same results without assistance or through other mechanisms. The grants and contributions often filled a funding gap and leveraged limited financial resources to achieve enormous gains in support of stakeholder objectives. The CSA is able to provide leadership in the space sector that positions it as an enabler in focusing research on concrete results.

Virtually all interviewees stated that assistance was required to achieve results. While the grants and contributions rarely covered all project expenses, the assistance provided was typically leveraged with other funding sources to ensure success that would otherwise have been unachievable.

Annex I is the exception in this regard. Management interviewees indicated that while supplements for Annex I recipients were most welcome, students were already planning on pursuing space-related studies and therefore funds may not have been required. Recipients indicated that these supplements did show good will on behalf of the CSA, but that the funding had not influenced their choice of career paths.

3.6.2 Alternatives

Conclusion

CSA financial assistance has been instrumental to most participants in leveraging additional funding for their projects which constitutes the most cost-effective way to achieve the Program's intended results. There are opportunities for CSA to increase the efficiency of service delivery by streamlining the number of Annexes and by centralizing some of the administrative aspects of the Program.

Interviewees and focus group participants were nearly unanimous in their praise of the Class G&C Program and were not able to suggest more cost effective ways to achieve the same results. Neither management nor recipients were able to suggest better ways to achieve the same results and stated that many of the projects would not have been achieved without funding.

Several management interviewees did voice concerns over the lack of coordination between program managers and thought that more centralization of G&C administration would be appropriate. They indicated that some centralization would be appropriate as it would lessen the administrative work needed and ensure a more effective administration of the Program.

4. Overall Conclusions

This section presents the overall conclusions from the evaluation of the G&C Program by issue area.

4.1 Relevance

The Class G&C Program continues to be relevant as evidenced by its alignment with Government of Canada priorities, the objectives of the Canadian Space Strategy and the mandate of the CSA. There is a continued need for the Class G&C Program to ensure innovative capacity and human resources capacity in the Canadian space sector. The CSA is the only organization that is poised to offer focused support to stakeholders in achieving these objectives. The Class G&C Program provides the financial support in areas in which other sources of funding do not exist.

4.2 Success – Awareness Component

The awareness component of the Class G&C Program appropriately targets learning organization animators, educators and Canadian students at the primary, secondary and post-secondary levels. The methods of delivery, although different between the grant and contribution annexes, are focused on the right types of intervention. The Program appears to have reached a large number of individuals, based on the funding of forty-six projects and the awarding of grants to one hundred and sixty individuals to date. Anecdotal evidence suggests that participants are gaining a greater awareness of S&T through their exposure to topics of interest in the Canadian space sector.

4.3 Success – Human Resources Component

Recipients are gaining relevant expertise and knowledge from the human resources component of the Program. Recipients are also very active in transmitting their expertise to CSA partners, either through academic means, such as research papers and conference presentations, or through workshops. A significant number of recipients have gained employment in the Canadian space sector as a result of the Program. Many grant recipients from Annex I would have pursued a space-related program even in the absence of funding.

4.4 Success – Research Component

The research component supports the generation of research results that are useful and relevant to the Canadian space sector. This research creates technical know-how in Canada that, while not always immediately applicable, does create a foundation which can either be utilized in future space missions or transferred to terrestrial issues. Recipients are only beginning to disseminate research results due to the relatively young age of the Program and the time required to complete the research projects. Given the long lead times required to plan and launch space missions, it is not reasonable to expect that research results generated by the Program are being applied to the Canadian Space Program. However, there is some evidence that research at an advanced stage of completion is being utilized within the Canadian space sector.

4.5 Success – Other

The Class G&C Program has fostered a collaborative atmosphere in the Canadian space sector. Many of the Annexes are designed specifically to include some form of networking, allowing space sector participants an opportunity to work together on projects, share information and develop the basis for future collaboration. A number of interviewees and focus group participants noted that the increased collaboration carried with it a cost in terms of time and money and that the objectives and agendas of all participants need to be considered.

CSA has addressed or is in the process of addressing all issues raised in a mid-term review of the Program. However, the Program's current governance structure and performance measurement framework are inadequate to clearly demonstrate accountability for and achievement of results. CSA management is already engaged in a renewal process that will include new Terms and Conditions and a revised Performance Measurement Strategy for the Program.

4.6 Cost-Effectiveness

Most recipients would not have been able to achieve their results without financial assistance from the Program, while other funding sources are either non-existent or would be much less effective. Funding was deemed unnecessary for Annex I as it does not attract additional participants to the space sector. The Class G&C Program is a cost effective method for supporting space program objectives, however, there are opportunities for CSA to increase the efficiency of service delivery by streamlining the number of Annexes and by centralizing some of the administrative aspects of the Program.

5. Recommendations

This section of the report provides recommendations for the Program based on the findings and conclusions reported in section 3.

5.1 Streamline Program Components

The twelve Annexes of the Program should be streamlined into three primary components that focus on increasing the awareness, human resources and research capacity of the Canadian space sector. CSA should abandon those Annexes that have yet to be implemented (Annexes V and XI) as well as Annex I which is not meeting expectations with regard to the achievement of results in the most cost-effective manner. Rationalizing the Program in this manner would permit a more targeted allocation of G&C funding in line with the needs of recipients while ensuring that missed opportunities caused by a shortage of funding are minimized. By focusing in on the areas of highest priority for the CSA and its partners, a streamlined Program would have a positive influence on overall impacts and cost-effectiveness.

5.2 Broaden the Scope of the HR Component

The scope of the human resources component of the Program needs to be broadened to better meet the long term requirements of the Canadian space sector. Numerous respondents indicated the requirement to develop human resources capacity in other functional areas, such as engineering and technology, by spreading the funding for human resources development beyond the traditional emphasis on doctoral and post-doctoral researchers. The retargeting of funding for the HR component should be based on a more detailed assessment of the human resources needs of the Canadian space sector, which clearly goes beyond the scope of this evaluation.

5.3 Centralize Program Administration

In addition to streamlining Program components, additional benefits could be derived by centralizing certain administrative aspects of service delivery such as calls for proposal, records management, performance reporting, etc. Increased efficiencies could be achieved by designating a program office for the Class G&C Program and potentially other grant and contribution programs at CSA. A support function of this nature might contribute to reduced administrative burden, decreased duplicity of effort and increased standardization of processes without removing the accountability for results from the appropriate technical authorities.

5.4 Improve Accountability for Results

In renewing the Class Grant and Contribution Program, CSA should pay particular attention to improving accountability for the achievement of results through the development of an updated governance structure and performance measurement strategy. A simplified governance structure would help to clarify accountabilities in program delivery, especially in the case of projects involving multiple stakeholders from within and beyond the CSA. The improved quality and availability of results-based performance information would greatly assist management in monitoring the Program's operational performance while clearly demonstrating the extent to which key expected results are being achieved. These improvements would significantly enhance the confidence of central agencies, departmental stakeholders and Canadians in the CSA's ability to manage for results.

Appendix A: Description of Annexes

  • Annex ICSA Supplements to NSERC Post-Graduate Scholarships and to Post-Doctoral Fellowships: Fosters advanced studies in space science and technology by offering a supplement equal to one-third of the value of the Natural Sciences and Engineering Research Council (NSERC) Postgraduate Scholarships (PGS), Canada Graduate Scholarships (CGS) and Postdoctoral Fellowships (PDF).
  • Annex IICSA Fellowships in Space Science, Space Technology and Aerospace Medicine: Encourages promising space science and technology researchers and medical specialists to conduct space-related activities in a university, an industry or a government research institution in Canada.
  • Annex IIICSA Support to NSERC Partnerships Programs: Fosters advanced research in space science and technology areas and training of highly qualified personnel by encouraging R&D collaboration between space industries and universities. CSA, NSERC and industry share the direct costs of research projects carried out in universities.
  • Annex IVCSA Support to NSERC Industrial Research Chairs: Provides funding for research chairs implemented with university and industry collaborations to develop Canadian advanced expertise in space science and technologies and support the training of highly qualified personnel. CSA, NSERC and industry share the direct costs of research chairs.
  • Annex V – Support to Not-For-Profit R&D Organizations for Space Research and Technology Development: Shares the costs of pre-competitive research projects in space-related areas of interest to Canada to develop advanced technologies for future space missions and Canadian Space Program priorities. Not used during this review.
  • Annex VI – Aerospace Medicine Elective Award: Encourages medical students and residents in the area of clinical, operational and research aspects of space medicine to pursue interests and careers in aerospace medicine by facilitating attendance at NASA's training sessions at the Johnson and Kennedy Space Centers.
  • Annex VII – Space Science Enhancement Program: Provides funding to university scientists for conducting initial concept studies on space science instruments, carrying out analysis on data produced by space science missions, and performing space science-related academic studies.
  • Annex VIIICSA Support to Scientific and Technical Space Conferences: Facilitates the diffusion of space scientific knowledge and technologies among stakeholders, with specialized audiences and/or non-space-related influential fora by supporting attendance at conferences, workshops, and seminars on space-related topics of interest for the Canadian Space Program.
  • Annex IX – Youth Space Awareness Contributions Program: Facilitates access to opportunities for learning about space science and technology to improve scientific and technological literacy among students and educators to encourage students to pursue studies and careers in science, engineering and mathematics, to promote and diffuse space knowledge to enhance the interest and awareness of Canadians, to consolidate Canada's expertise in leading space niches by building a qualified workforce and to instil a sense of national pride.
  • Annex XCSA Support to Institutions Dedicated to Space Research and Education: Provides young Canadians with multi-disciplinary and international training in space-related studies through financial support of the International Space University (ISU), the Canadian Foundation for ISU (CFISU), the International Program Office (IPO) for the Stratospheric Processes and their Role in Climate (SPARC) Project and the Global Forum on Space Economics of the Organization for Economic Co-operation and Development (OECD).
  • Annex XI – Joint CSA / Network Centres of Excellence Research Program: Supports advanced space research in universities in areas of priority to Canada and facilitates the transfer of university-generated knowledge and technology to industry through strategic alliances and research partnerships between industry, universities and CSA. Not used during this review.
  • Annex XII – Space Awareness and Learning Grant Program: Encourages the participation of youth and educators in educational events designed to improve scientific and technological literacy among students and educators to encourage students to pursue studies and careers in science, engineering and mathematics, to promote and diffuse space knowledge to enhance the interest and awareness of Canadians, to consolidate Canada's expertise in leading space niches by building a qualified workforce and to instil a sense of national pride.

Appendix B: Summary of Governance Structure

Annex Space Tech. Space Science CAO Comm. ER NSERC Recipients
I Accountable for Program Management Accountable for Program Management       Accountable for Performance Measurement  
II Accountable for Program Management Accountable for Program Management Accountable for Program Management        
III Accountable for Program Management Accountable for Program Management and Performance Measurement       Accountable for Performance Measurement  
IV Accountable for Program Management Accountable for Program Management       Accountable for Performance Measurement  
V Accountable for Program Management           Accountable for Performance Measurement
VI     Accountable for Program Management        
VII   Accountable for Program Management          
VIII Accountable for Program Management Accountable for Program Management          
IX       Accountable for Program Management      
X         Accountable for Program Management    
XI Accountable for Program Management           Accountable for Performance Measurement
XII       Accountable for Program Management      
Legend
  • cercle vide Accountable for Program Management
  • cercle demi plein Accountable for Performance Measurement
  • cercle plein Accountable for Program Management and Performance Measurement

Appendix D: Evaluation Matrix

Evaluation Issues / Questions Performance Indicators Data Sources Methodologies
Rationale / Relevance
1. Does the Program continue to address an ongoing need for which it was originally established?
  • Assessment of past / present innovative capacity in space sector
  • Assessment of past / present human resources capacity in space sector
  • Economic data for the space sector
  • Reports on research capabilities
  • Draft Memoranda to Cabinet
  • Draft Treasury Board Submissions
  • Interview findings
  • Focus group findings
  • Document review
  • Interviews with program managers
  • Focus group with advisory committee
  • Focus group with CSA partners
2. Is the Program aligned with the mandate of the Canadian Space Agency, the Canadian Space Strategy and Government of Canada priorities?
  • Alignment of Program outcomes with CSA mandate
  • Alignment of Program outcomes with Canadian Space Strategy objectives
  • Alignment of Program outcomes with current GoC priorities
  • Canadian Space Agency Act
  • Canadian Space Strategy
  • PMO / Clerk / Minister's website
  • Draft Memoranda to Cabinet
  • Draft Treasury Board Submissions
  • Interview findings
  • Document review
  • Interviews with program managers
Success – Awareness Component
3. Is the correct group being targeted by the awareness component of the Program and is the method of intervention appropriate to their needs?
  • Assessment of the appropriateness of the target group based on the needs of CSA
  • Assessment of the appropriateness of the intervention based on the needs of the target group
  • Year-end performance reports from program managers
  • Departmental Performance Report / Annual Report
  • Surveys of participants in educational events (may not exist)
  • Interview findings
  • Document review
  • Interviews with program managers
  • Interviews with Recipients
4. To what extent does the awareness component of the Program reach young Canadians?
  • % of target population reached by the Program
  • Year-end performance reports from program managers
  • Departmental Performance Report / Annual Report
  • Surveys of participants in educational events (may not exist)
  • Interview findings
  • Document review
  • Interviews with program managers
  • Interviews with recipients
5. Do young Canadians have a greater awareness of S&T as a result of the awareness component of the Program?
  • % of participants in educational events claiming increased awareness of S&T (may not exist)
  • Year-end performance reports from program managers
  • Departmental Performance Report / Annual Report
  • Surveys of participants in educational events (may not exist)
  • Interview findings
  • Document review
  • Interviews with program managers
  • Interviews with recipients
Success – Human Resources Component
6. Are recipients acquiring relevant expertise from the HR component of the Program?
  • Learning projects achieve objectives set out in G&C agreements
  • Learning project results are aligned with areas of priority to Canada
  • # scholarships, fellowships industrial chairs, training sessions funded
  • # individuals attending ISU, conferences, seminars and workshops
  • # Recipients obtaining Masters, PhD or Post-Doctorate degrees
  • # Recipients obtaining board certification in aerospace medicine
  • Year-end performance reports from program managers
  • Departmental Performance Report / Annual Report
  • Program terms and conditions and eligibility criteria
  • Interview findings
  • Focus group findings
  • Document review
  • Interviews with program managers
  • Interviews with recipients
  • Focus group with advisory committee
  • Focus group with CSA partners
7. Are recipients transmitting their expertise to CSA partners who were not involved in the learning project?
  • # recipients teaching, training, conferences, seminars, workshops
  • # recipients involved in collaborative learning projects
  • # learning projects involving one, two, three or more CSA partners
  • Documents TBD
  • Interview findings
  • Focus group findings
  • Document review
  • Interviews with program managers
  • Interviews with recipients
  • Focus group with advisory committee
  • Focus group with CSA partners
8. Are recipients gaining employment in the Canadian space sector or other scientific disciplines, either domestically or abroad?
  • Recipients are employed by CSA and its partners
  • Recipients working on current and future missions
  • Recipients working on current and future mission concepts
  • Economic data for the space sector
  • Documents TBD
  • Interview findings
  • Focus group findings
  • Document review
  • Interviews with program managers
  • Interviews with recipients
  • Focus group with advisory committee
  • Focus group with CSA partners
Success – Research Component
9. Are recipients generating useful research results from the research component of the Program?
  • Research projects achieve objectives set out in G&C agreements
  • Research results are aligned with areas of priority to Canada
  • Research results permit or exclude new avenues of research / application
  • Year-end performance reports from program managers
  • Departmental Performance Report / Annual Report
  • Program terms and conditions and eligibility criteria
  • Interview findings
  • Focus group findings
  • Document review
  • Interviews with program managers
  • Interviews with recipients
  • Focus group with advisory committee
  • Focus group with CSA partners
10. Are recipients transmitting research results to CSA partners who were not involved in the research project?
  • CSA partners demonstrate an awareness of research results
  • Research results published in journals, white papers, etc.
  • Research results presented to conferences, seminars, etc.
  • Documents TBD
  • Interview findings
  • Focus group findings
  • Document review
  • Interviews with program managers
  • Interviews with recipients
  • Focus group with advisory committee
  • Focus group with CSA partners
11. Are research results being applied to the Canadian Space Program or other scientific disciplines, either domestically or abroad?
  • Research results cited by space scientists and in other disciplines
  • Research results incorporated into successive research projects
  • Research results applied to current and future mission concepts
  • Research results applied to current and future missions
  • Research results generate patents or licenses
  • Documents TBD
  • Interview findings
  • Focus group findings
  • Document review
  • Interviews with program managers
  • Interviews with recipients
  • Focus group with advisory committee
  • Focus group with CSA partners
Success – Other
12. Has the Program fostered increased collaboration between stakeholders in the Space Sector that has helped them to better achieve mutual objectives?
  • Evidence of the impacts of collaboration
  • Year-end performance reports from program managers
  • Departmental Performance Report / Annual Report
  • Financial data
  • Interview findings
  • Focus group findings
  • Results of comparative analysis
  • Document review
  • Interviews with program managers
  • Interviews with recipients
  • Focus group with advisory committee
  • Focus group with CSA partners
  • Comparative analysis of like programs
13. Has the Program resulted in any unintended impacts (positive or negative)?
  • Evidence of unintended impacts
  • Documents TBD
  • Interview findings
  • Focus group findings
  • Document review
  • Interviews with program managers
  • Interviews with recipients
  • Focus group with advisory committee
  • Focus group with CSA partners
14. Have issues addressed in previous audits, reviews and evaluations been addressed?
  • Evidence that audit, review and evaluation observations have been accepted and acted upon
  • Previous audits, reviews and evaluations
  • Management response to previous audits, reviews and evaluations
  • Interview findings
  • Document review
  • Interviews with program managers
Cost-effectiveness – Alternatives
15. Would recipients have achieved the same results from their projects without financial assistance from the Program?
  • TBD
  • Year-end performance reports from program managers
  • Departmental Performance Report / Annual Report
  • Financial data
  • Interview findings
  • Focus group findings
  • Results of comparative analysis
  • Document review
  • Interviews with program managers
  • Interviews with recipients
  • Focus group with advisory committee
  • Focus group with CSA partners
  • Comparative analysis of like programs
16. Are there more cost-effective ways to achieve the same outcomes as the Program?
  • TBD
  • Year-end performance reports from program managers
  • Departmental Performance Report / Annual Report
  • Financial data
  • Interview findings
  • Focus group findings
  • Results of comparative analysis
  • Document review
  • Interviews with program managers
  • Interviews with recipients
  • Focus group with advisory committee
  • Focus group with CSA partners
  • Comparative analysis of like programs

Appendix E: Strategic Review Crosswalk

No. Strategic Review Question Evaluation Question
1 Are all programs and spending effectively aligned to the government's priorities? Is the Program aligned with the mandate of the Canadian Space Agency, the Canadian Space Strategy and Government of Canada priorities?
2 Are all programs and spending consistent with federal roles and responsibilities? Is the Program aligned with the mandate of the Canadian Space Agency, the Canadian Space Strategy and Government of Canada priorities?
3 Are there other organizations (federal/provincial, private sector, or non-profit sector) better placed to deliver these programs? Is the Program aligned with the mandate of the Canadian Space Agency, the Canadian Space Strategy and Government of Canada priorities?
4 Are all programs and spending relevant and still meeting the needs of Canadians? Does the Program continue to address an ongoing need for which it was originally established?
5 Are all programs maximizing value for money? Would recipients have achieved the same results from their projects without financial assistance from the Program?
6 Are all programs effectively serving the purpose for which they were created? Do young Canadians have a greater awareness of S&T as a result of the awareness component of the Program? Are recipients gaining employment in the Canadian space sector or other scientific disciplines, either domestically or abroad? Are research results being applied to the Canadian Space Program or other scientific disciplines, either domestically or abroad?
7 Are adjustments required to improve performance (including efficiency and effectiveness)? Do young Canadians have a greater awareness of S&T as a result of the awareness component of the Program? Are recipients gaining employment in the Canadian space sector or other scientific disciplines, either domestically or abroad? Are research results being applied to the Canadian Space Program or other scientific disciplines, either domestically or abroad?
8 Can we achieve the same results for less money? Are there more cost-effective ways to achieve the same outcomes as the Program?
9 Does the organization have the capacity to effectively spend and manage for results? Do young Canadians have a greater awareness of S&T as a result of the awareness component of the Program? Are recipients gaining employment in the Canadian space sector or other scientific disciplines, either domestically or abroad? Are research results being applied to the Canadian Space Program or other scientific disciplines, either domestically or abroad?
10 Are adjustments needed to improve management performance? Have issues addressed in previous audits, reviews and evaluations been addressed?

Appendix F: List of Documents Reviewed

Secondary Research for Conceptual Framework

  1. Measuring and Reporting Soft Outcomes, Treasury Board of Canada, Secretariat, November 2003.
  2. Monica Salazar-Acosta, "Innovation systems based indicators: Emphasis on human capital and ICT adoption," Simon Fraser University, 2006.

Guidance on Grants and Contributions

  1. Policy on Transfer Payments, Treasury Board of Canada, Secretariat (2000).
  2. Guide on Grants, Contributions and Other Transfer Payments, Treasury Board of Canada, Secretariat (2002).
  3. Draft Directive on Transfer Payments, Treasury Board of Canada, Secretariat (2008).
  4. From Red Tape to Clear Results: The Report of the Independent Blue Ribbon Panel on Grant and Contribution Program, Government of Canada (2006).
  5. The Government of Canada Action Plan to Reform the Administration of Grant and Contribution Programs, Treasury Board of Canada, Secretariat (2008).

Policy Drivers for CSA

  1. Canadian Space Agency Act, 1990, c. 13, Department of Justice (1990).
  2. The Canadian Space Strategy: Serving and Inspiring the Nation, Canadian Space Agency (2003).
  3. National Aerospace and Defence Strategic Framework: 2005-2025, Industry Canada (2005).
  4. The Global Exploration Strategy: the Framework for Coordination, ASI, BNSC, CNES, CNSA, CSA, CSIRO, DLR, ESA, ISRO, JAXA, KARI, NASA, NSAU, Roscosmos (2007).
  5. Canada's New Government: Mobilizing Science and Technology to Canada's Advantage, Government of Canada (2007).
  6. Speaking Points, The Honourable Jim Prentice, PC, QC, MP, Minister of Industry, Canadian Space Agency, St. Hubert, Quebec (April 11, 2008).
  7. Speaking Points, The Honourable Jim Prentice, PC, QC, MP, Minister of Industry, Announcement of NSERC's Research Grants and Scholarships, Victoria, British Columbia (May 21; 2008).
  8. Speaking Points, The Honourable Jim Prentice, PC, QC, MP, Minister of Industry, Canadian Space Agency Address, Saint-Hubert, Quebec (May 9, 2008).
  9. Speaking Points, The Honourable Jim Prentice, PC, QC, MP, Minister of Industry, Canadian Space Agency, Canadian Astronaut Recruitment Campaign Announcement, Longueuil, Quebec (March 31, 2008).

CSA Corporate Reports

  1. Report on Plans and Priorities (RPP), Canadian Space Agency (2004/2005 - 2008/2009).
  2. Departmental Performance Report (DPR), Canadian Space Agency (2003/2004 - 2007/2008).
  3. Management Accountability Framework (MAF) Assessment 05/06, Canadian Space Agency (2006).
  4. Management Accountability Framework (MAF) Assessment 05/06, Treasury Board of Canada, Secretariat (2006).
  5. Modern Comptrollership Evaluation, Canadian Space Agency, Audit and Evaluation Directorate (2003).
  6. Report of the Auditor General of Canada to the House of Commons: Chapter 7: Canadian Space Agency - Implementing the Canadian Space Program, Office of the Auditor General Of Canada (2002).
  7. Assessment of CSA Capacity in the Light of the 10 expectations of the Management Accountability Framework for the Fiscal Year 2005-06, Canadian Space Agency, (2008)

Program-Related Documentation

  1. Draft Treasury Board Submissions for the Class Grant and Contribution Program, Treasury Board of Canada, Secretariat (2002, 2005, 2007).
  2. Terms and Conditions of Annexes I-XII, Canadian Space Agency Website (2008).
  3. Audit Report: Grant and Contribution Program to Support Awareness, Research and Training in Space Science and Technology, Project #04/05 01-02, Canadian Space Agency, Audit, Evaluation and Review Directorate (2005).
  4. Mid-Term Review of Management and Performance of the Class Grant and Contribution Programs to Support Awareness, Research Training in Space Science and Technology, Audit Services Canada (2007).
  5. Suivi du Plan D'action de la Gestion au 31 Décembre 2007 Concernant le Projet de Vérification, Canadian Space Agency (2008).
  6. Terms of Reference for the CSA Science Advisory Committees, Canadian Space Agency (2007).
  7. Terms of Reference for the Senior Space Science Advisory Committee, Canadian Space Agency (2007).
  8. Minutes and Records of Decision from Senior Space Science Advisory Committee re: Priority Setting and Recommendations for Various Annexes
  9. Minutes and Records of Decision from CSA Science Advisory Committees re: Priority Setting and Recommendations for Various Annexes
  10. Sommaire de plan de travail
    Secteur : Communications et affaires publiques
    2007/2008, Canadian Space Agency, (2008).

Program-Related Performance Information

  1. 2003-2006 CSA Class Grant and Contribution Program, "A Statistical Performance Survey and Internal Evaluation of Annexes I, II, IV, VII, VIII, and X", Athena Global (2007).
  2. Excel Spreadsheets for 2007 (Annexes I, II, IV, VII, VIII, and X), Athena Global (2008).
  3. Partnerships Support Program Annual Report (2005-2006, 2006-2007, 2007-2008), Canadian Space Agency (2008).
  4. Aerospace Medicine Training - Program Report, Canadian Space Agency (2008).
  5. Survey and annual reports and list of data sources for Annex IX, Canadian Space Agency (2008).
  6. Summary report and Excel Spreadsheets for 2007and 2008 recipients, report on OECD component (Annex X), Canadian Space Agency, 2008.
  7. Supplements to NSERC Postgraduate Scholarships and Postdoctoral Fellowships - Evaluation Report (2003, 2004, 2005), Canadian Space Agency, 2008.

Appendix G: List of Individuals Interviewed

Senior Management

  1. Paul Engel
    Director, Communications and Public Affairs
    Communications and Public Affairs
    Canadian Space Agency
  2. Hugues Gilbert
    Director, Policy and External Relations
    Policy and External Relations
    Canadian Space Agency
  3. Gilles Leclerc
    Director General, Space Technologies
    Space Technologies
    Canadian Space Agency
  4. David Kendall
    Director General, Space Science
    Space Science
    Canadian Space Agency

Program Management

  1. Jason Clement
    Communications Officer, Space Awareness and Learning
    Communications and Public Affairs
    Canadian Space Agency
  2. Michel Doyon
    Manager, Science and Exploration Technologies
    Technology Management and Applications
    Canadian Space Agency
  3. Joan Harvey
    Head, Research and Analysis
    Policy and External Relations
    Canadian Space Agency
  4. Natalie Hirsch
    Project Officer
    Operational Space Medicine
    Canadian Space Agency
  5. Denis Laurin
    Sr. Program Scientist, Planetary Exploration and Space Astronomy
    Planetary Exploration and Space Astronomy
    Canadian Space Agency
  6. Martin Lebeuf
    Manager, Applied Science
    Planetary Exploration and Space Astronomy
    Canadian Space Agency
  7. Patricia Marshall
    Consultant
    Space Science and Space Technologies
    Canadian Space Agency
  8. Thu-Oahn Nguyen
    Manager, Space Science Development
    Planetary Exploration and Space Astronomy
    Canadian Space Agency
  9. Marilyn Steinberg
    Program Manager,
    Space Awareness and Learning
    Communications and Public Affairs
    Canadian Space Agency

Grant & Contribution Recipients

  • Annex I (Human Resources)
  • Annex II (Human Resources)
  • Annex III (Research)
  • Annex IV (Human Resources)
  • Annex VI (Human Resources)
  • Annex VII (Research)
  • Annex VIII (Human Resources)
  • Annex IX (Awareness)
  • Annex XII (Awareness)

Appendix H: Interview Guides

Interview Guide for Management Interviews – Awareness

Overview

Government Consulting Services (GCS) has been engaged by the Canadian Space Agency (CSA) to undertake a summative evaluation of the Class Grant and Contribution (G&C) Program. The evaluation focuses on issues of program relevance, success and cost-effectiveness.

One of the main lines of enquiry for this evaluation involves interviews with key stakeholders in the Class G&C Program. To this end, GCS is conducting interviews with CSA management staff as well as the recipients of grants and contributions. The following questions serve as a guide for the interviews with CSA management staff.

Please note that your individual responses are confidential and that the final evaluation report will only present the interview findings in aggregate form.

Background

  1. Please describe your role as it relates to the Program.

Relevance Questions

  1. What needs were the Program designed to fulfil?
  2. Are those needs still relevant today?
  3. Does the Program continue to address those needs?
  4. How does the Program contribute to:
    1. The mandate of the Canadian Space Agency?
    2. The objectives of the Canadian Space Strategy?
    3. The priorities of the Government of Canada?

Success Questions – Awareness Component

  1. Are young Canadians the most appropriate beneficiaries of the Program?
  2. Does the method of delivery meet the needs of young Canadians?
  3. To what extent does the Program reach young Canadians?
  4. What evidence exists to suggest that young Canadians have a greater awareness of science and technology as a result of the Program?
  5. Has the Program resulted in any unintended or unexpected positive impacts or negative consequences?
  6. What changes have been made to the Program to address issues raised by previous audits, reviews and evaluations?

Cost-Effectiveness Questions

  1. What types of synergies have resulted from the requirement to collaborate with one or more partners under the Program?
  2. Would recipients have achieved the same results from their projects without financial assistance from the Program?
  3. Are there other ways to achieve the same results as the Program in a more cost-effective manner?
  4. What improvements would you recommend to make the Program more cost-effective?

Closing Questions

  1. Are there any other issues that we have not covered but you feel are important?

Interview Guide for Management Interviews – HR

Overview

Government Consulting Services (GCS) has been engaged by the Canadian Space Agency (CSA) to undertake a summative evaluation of the Class Grant and Contribution (G&C) Program. The evaluation focuses on issues of program relevance, success and cost-effectiveness.

One of the main lines of enquiry for this evaluation involves interviews with key stakeholders in the Class G&C Program. To this end, GCS is conducting interviews with CSA management staff as well as the recipients of grants and contributions. The following questions serve as a guide for the interviews with CSA management staff.

Please note that your individual responses are confidential and that the final evaluation report will only present the interview findings in aggregate form.

Background

  1. Please describe your role as it relates to the Program.

Relevance Questions

  1. What needs were the Program designed to fulfil?
  2. Are those needs still relevant today?
  3. Does the Program continue to address those needs?
  4. How does the Program contribute to:
    1. The mandate of the Canadian Space Agency?
    2. The objectives of the Canadian Space Strategy?
    3. The priorities of the Government of Canada?

Success Questions – Human Resources Component

  1. Have the beneficiaries of the Program acquired expertise that is relevant to the needs of the CSA or the Canadian Space Sector?
  2. How have the beneficiaries of the Program shared their expertise with relevant stakeholders in the CSA or the Canadian Space Sector?
  3. How have these stakeholders benefited from the transmission of expertise from the beneficiaries of the Program?
  4. What evidence suggests that beneficiaries of the Program have gained employment in the Canadian Space Sector or other scientific disciplines?
  5. Has the Program resulted in any unintended or unexpected positive impacts or negative consequences?
  6. What changes have been made to the Program to address issues raised by previous audits, reviews and evaluations?

Cost-Effectiveness Questions

  1. What types of synergies have resulted from the requirement to collaborate with one or more partners under the Program?
  2. Would recipients have achieved the same results from their projects without financial assistance from the Program?
  3. Are there other ways to achieve the same results as the Program in a more cost-effective manner?
  4. What improvements would you recommend to make the Program more cost-effective?

Closing Questions

  1. Are there any other issues that we have not covered but you feel are important?

Interview Guide for Management Interviews – Research

Overview

Government Consulting Services (GCS) has been engaged by the Canadian Space Agency (CSA) to undertake a summative evaluation of the Class Grant and Contribution (G&C) Program. The evaluation focuses on issues of program relevance, success and cost-effectiveness.

One of the main lines of enquiry for this evaluation involves interviews with key stakeholders in the Class G&C Program. To this end, GCS is conducting interviews with CSA management staff as well as the recipients of grants and contributions. The following questions serve as a guide for the interviews with CSA management staff.

Please note that your individual responses are confidential and that the final evaluation report will only present the interview findings in aggregate form.

Background

  1. Please describe your role as it relates to the Program.

Relevance Questions

  1. What needs were the Program designed to fulfil?
  2. Are those needs still relevant today?
  3. Does the Program continue to address those needs?
  4. How does the Program contribute to:
    1. The mandate of the Canadian Space Agency?
    2. The objectives of the Canadian Space Strategy?
    3. The priorities of the Government of Canada?

Success Questions – Research Component

  1. Have the beneficiaries of the Program generated research results that are relevant to the needs of the CSA or the Canadian Space Sector?
  2. How have the beneficiaries of the Program shared their research results with relevant stakeholders in the CSA or the Canadian Space Sector?
  3. How have these stakeholders benefited from the transmission of research results from the beneficiaries of the Program?
  4. What evidence suggests that the research results generated by the Program have been applied by the Canadian Space Sector or other scientific disciplines?
  5. Has the Program resulted in any unintended or unexpected positive impacts or negative consequences?
  6. What changes have been made to the Program to address issues raised by previous audits, reviews and evaluations?

Cost-Effectiveness Questions

  1. What types of synergies have resulted from the requirement to collaborate with one or more partners under the Program?
  2. Would recipients have achieved the same results from their projects without financial assistance from the Program?
  3. Are there other ways to achieve the same results as the Program in a more cost-effective manner?
  4. What improvements would you recommend to make the Program more cost-effective?

Closing Questions

  1. Are there any other issues that we have not covered but you feel are important?

Interview Guide for Recipient Interviews – Awareness

Overview

Government Consulting Services (GCS) has been engaged by the Canadian Space Agency (CSA) to undertake a summative evaluation of the Class Grant and Contribution (G&C) Program. The evaluation focuses on issues of program relevance, success and cost-effectiveness.

One of the main lines of enquiry for this evaluation involves interviews with key stakeholders in the Class G&C Program. To this end, GCS is conducting interviews with CSA management staff as well as the recipients of grants and contributions. The following questions serve as a guide for the interviews with the recipients of grants and contributions.

Please note that your individual responses are confidential and that the final evaluation report will only present the interview findings in aggregate form.

Background

  1. Please describe your role as it relates to the Program.

Success Questions

  1. Are young Canadians the most appropriate beneficiaries of the Program?
  2. Does the method of delivery meet the needs of young Canadians?
  3. To what extent has the Program reached young Canadians?
  4. What evidence exists to suggest that young Canadians have a greater awareness of science and technology as a result of the Program?
  5. Has the Program increased the extent to which you collaborate with other partners in the Space Sector?
  6. How have these partnerships helped you and your partners to achieve mutual objectives?
  7. Are you aware of any unexpected impacts or negative consequences arising from your participation in the Program?

Cost-Effectiveness Questions

  1. Would you have achieved the same results from your projects without financial assistance from the Program?
  2. Are there other ways to achieve the same results as the Program in a more cost-effective manner?

Closing Questions

  1. Are there any other issues that we have not covered but you feel are important?

Interview Guide for Recipient Interviews – HR

Overview

Government Consulting Services (GCS) has been engaged by the Canadian Space Agency (CSA) to undertake a summative evaluation of the Class Grant and Contribution (G&C) Program. The evaluation focuses on issues of program relevance, success and cost-effectiveness.

One of the main lines of enquiry for this evaluation involves interviews with key stakeholders in the Class G&C Program. To this end, GCS is conducting interviews with CSA management staff as well as the recipients of grants and contributions. The following questions serve as a guide for the interviews with the recipients of grants and contributions.

Please note that your individual responses are confidential and that the final evaluation report will only present the interview findings in aggregate form.

Background

  1. Please describe your role as it relates to the Program.

Success Questions

  1. Has the Program helped you to acquire expertise that is relevant to the needs of the CSA or the Canadian Space Sector?
  2. How have you shared your expertise with relevant stakeholders in the CSA or the Canadian Space Sector?
  3. How have these stakeholders benefited from you transmitting your expertise to them?
  4. How has this expertise helped you to become more involved in the Canadian Space Sector or other scientific disciplines?
  5. Has the Program increased the extent to which you collaborate with other partners in the Space Sector?
  6. How have these partnerships helped you and your partners to achieve mutual objectives?
  7. Are you aware of any unexpected impacts or negative consequences arising from your participation in the Program?

Cost-Effectiveness Questions

  1. Would you have achieved the same results from your projects without financial assistance from the Program?
  2. Are there other ways to achieve the same results as the Program in a more cost-effective manner?

Closing Questions

  1. Are there any other issues that we have not covered but you feel are important?

Interview Guide for Recipient Interviews – Research

Overview

Government Consulting Services (GCS) has been engaged by the Canadian Space Agency (CSA) to undertake a summative evaluation of the Class Grant and Contribution (G&C) Program. The evaluation focuses on issues of program relevance, success and cost-effectiveness.

One of the main lines of enquiry for this evaluation involves interviews with key stakeholders in the Class G&C Program. To this end, GCS is conducting interviews with CSA management staff as well as the recipients of grants and contributions. The following questions serve as a guide for the interviews with the recipients of grants and contributions.

Please note that your individual responses are confidential and that the final evaluation report will only present the interview findings in aggregate form.

Background

  1. Please describe your role as it relates to the Program.

Success Questions

  1. Has the Program helped you to generate research results that are relevant to the needs of the CSA or the Canadian Space Sector?
  2. How have you shared your research results with relevant stakeholders in the CSA or the Canadian Space Sector?
  3. How have these stakeholders benefited from you transmitting your research results to them?
  4. How have your research results been applied by the Canadian Space Sector or other scientific disciplines?
  5. Has the Program increased the extent to which you collaborate with other partners in the Space Sector?
  6. How have these partnerships helped you and your partners to achieve mutual objectives?
  7. Are you aware of any unexpected impacts or negative consequences arising from your participation in the Program?

Cost-Effectiveness Questions

  1. Would you have achieved the same results from your projects without financial assistance from the Program?
  2. Are there other ways to achieve the same results as the Program in a more cost-effective manner?

Closing Questions

  1. Are there any other issues that we have not covered but you feel are important?

Appendix I: Focus Group Participants

  • Focus Group 1 - September 30, 1 - 4 pm EST
  • Focus Group 2 - October 2, 12 - 3 pm EST

Footnotes

Footnote 1

No projects have been funded by this Annex.

Return to footnote 1 referrer

Footnote 2

Canada's Space Strategy, November 2003

Return to footnote 2 referrer

Footnote 3

Mobilizing Science and Technology to Canada's Advantage, 2007

Return to footnote 3 referrer

Footnote 4

Results taken from Communications Results2 document.

Return to footnote 4 referrer

Footnote 5

Mobilizing Science and Technology to Canada's Advantage, 2007

Return to footnote 5 referrer

Footnote 6

Results provided by Athena Global GC survey results

Return to footnote 6 referrer

Footnote 7

Mid-Term Review of Management and Performance of the Class Grant and Contribution Programs to Support Awareness, Research Training in Space Science and Technology, Audit Services Canada (ASC), 2005

Return to footnote 7 referrer