Follow-up to management action plans March 31, 2006

Download the PDF version (319 KB)

G&C Program to Support Awareness, Research and Training in Space S&T

Project - Follow-up 04/05 01-02

Audit, Evaluation and Review Directorate

March 2006

Implementation Summary

This follow-up report on the implementation of management action plans concludes the internal audit process and outlines the measures taken by various responsible entities in response to our findings and recommendations. According to the follow-up process in effect, management action plans are to be reviewed annually until they are fully implemented in order to assess the extent of implementation and keep the Audit Committee abreast of progress made.

At the request of the Audit Committee, this follow-up audit was prepared less than one year after the audit report was submitted to the Audit Committee, which differs from standard practice.

This report contains follow-up findings (as of March 31, 2006) related to the audit of the Grant and Contribution Program to Support Awareness, Research and Training in Space Science and Technology (G&C Program). The audit report and the action plan were submitted to and approved by the Audit Committee in June 2005. The findings and recommendations set out in this audit report cover six directorates and branches.

We believe that this report clearly shows that most of the directorates/branches have diligently implemented the recommendations that were drawn up. On the other hand, we noted that most of the directorates/branches had not implemented the recommendations related to performance measurement as per the Results-based Management and Accountability Framework (RMAF) before the due date indicated in the action plan.

It was also noted that between the time the audit follow-up process was launched (November 2005) and March 31, 2006, when these findings were reported, a good number of the recommendations were implemented following discussions with officials overseeing the G&C Program.

The following pages contain additional details of the progress achieved with respect to the action plan of each directorate/branch.

Communications and Public Affairs Directorate

Audit project objective

The purpose of the audit project was to assess the degree to which current systems, procedures and resources relative to all aspects of managing the Grant and Contribution Program to Support Awareness, Research and Training in Space Science and Technology are effective, efficient and economical.

The Risk-Based Audit Framework (RBAF) of this G&C Program states that an audit of the program, including both the grant and contribution components, as well as a review of the risk management plan, will be conducted in the second year of the program.

Recommendations

In June 2005, we reported that the Communications and Public Affairs Directorate had to pay special attention to the administrative files kept for each recipient to ensure a satisfactory audit trail, as required under the Policy on Transfer Payments of the Treasury Board Secretariat (TBS). We noted that improvements could be made with respect to supporting documents for payment claims and advance payment requests and the verification of claims and recipient eligibility.

We also noted that, despite the fact that most of the risks listed in the RBAF had not been systematically reviewed, appropriate mitigation measures had been implemented for most of the risks identified. As for the RMAF, we noted that, other than the database for compiling data related to contributions, no performance measurement was conducted with respect to RMAF objectives.

Implementation status

The Communications and Public Affairs Directorate has carried out nearly all of the elements in its action plan in a highly satisfactory manner. The only element in abeyance is related to the implementation of a data-gathering system to measure program performance. Management has already begun drafting questionnaires that will be sent to recipients to gather the information required to determine whether the program's RMAF objectives have been met. This initiative is expected to be completed in the coming months.

With regard to the other initiatives, management has changed a number of administrative practices to address the shortcomings noted.

Space Technologies Branch

Audit project objective

The purpose of the audit project was to assess the degree to which current systems, procedures and resources relative to all aspects of managing the Grant and Contribution Program to Support Awareness, Research and Training in Space Science and Technology are efficient, effective and economical.

The Risk-Based Audit Framework (RBAF) of this G&C Program states that an audit of the program, including both the grant and contribution components, as well as a review of the risk management plan will be conducted in the second year of the program.

Recommendations

In June 2005, we reported that the Space Technologies Branch generally kept administrative files to ensure a satisfactory audit trail, as required under the TBS Policy on Transfer Payments. We also noted that the procedure for verifying recipients' eligibility needed to be reviewed, that the NSERC should have been notified in writing regarding the payment of supplements, and that the $100,000 grant awarded to GEOIDE in 2004-2005 was not adequately supported by documentation as required under the grant agreement.

We also noted that, despite the fact the risks identified in the RBAF had not been systematically reviewed, a good number of mitigation measures had nevertheless been implemented in the daily management of the program. With regard to the RMAF, we noted that no data-gathering system had been implemented to measure program performance.

Implementation status

The Space Technologies Branch has implemented five of the seven recommendations in a satisfactory manner. The procedure for verifying the eligibility of recipients was reviewed and implemented for supplements paid out in 2005-2006.

With regard to the grant awarded to GEOIDE, a consultant is currently working on gathering the supporting documents related to the $100,000 grant. Our review of the preliminary report prepared by the consultant revealed that only $70,000 was spent on eligible projects.

This year, management has reviewed all of the risks and mitigation measures set out in the RBAF in order to ensure that they were still relevant and applied in the daily management of the program. Such a review will now be conducted on an annual basis.

With respect to performance measurement under the RMAF, a survey was attached to letters of agreement sent to supplement recipients this year. The survey questions are expected to elicit responses that will make it possible to measure the performance of the G&C Program component listed in Appendix I, namely CSA Supplements to NSERC Postgraduate Scholarships and Postdoctoral Fellowships, in accordance with RMAF objectives.

Management has decided not to commit itself to measure the performance of this G&C Program component for previous years (From April 1, 2003 to March 31, 2005).

Space Science Branch

Audit project objective

The purpose of the audit project was to assess the degree to which current systems, procedures and resources relative to all aspects of managing the Grant and Contribution Program to Support Awareness, Research and Training in Space Science and Technology are efficient, effective and economical.

The Risk-Based Audit Framework (RBAF) of this G&C Program states that an audit of the program, including both the grant and contribution components, as well as a review of the risk management plan will be conducted in the second year of the program.

Recommendations

In June 2005, we reported that the Space Science Branch generally kept administrative files to ensure a satisfactory audit trail, as required under the TBS Policy on Transfer Payments. We also noted that improvements could be made with respect to the procedure for verifying recipients' eligibility, compliance with certain approved terms and conditions of the Program and claim assessment.

Furthermore, the risks and mitigation measures identified in the RBAF had not been systematically reviewed. With respect to the RMAF, we noted that no data-gathering system had been implemented to measure program performance.

Implementation status

The Space Science Branch has implemented 7 of the 12 recommendations in a satisfactory manner. The procedure for verifying the eligibility of recipients was reviewed and implemented for supplements paid out in 2005-2006. Written communications with the NSERC confirmed the payment of supplements and decisions made following the assessment of payment claims related to the support program for NSERC industrial research chairs in order to comply with the approved terms and conditions.

Some actions under the Support Program for Scientific and Technical Space Conferences were not implemented on time as originally scheduled in the action plan. We reviewed the five files for which grants were awarded under this component of the G&C Program in 2005-2006 and noted the following:

  • Not all of the required letters of support from sponsor organizations were on file (one was missing, the other had been submitted after the event).
  • There was no evidence on file indicating that an evaluation was done regarding the extent to which the financial resources allocated to the event were suitable (it was only noted that a document related to costs and funding sources had been received).
  • One of the files reviewed included non-eligible costs, in this case, administrative costs.
  • In three of the five cases, the letters of agreement had been signed after the event.
  • In three of the five cases, the commitment of funds was made after the agreement had been signed. Of those three cases, two have been committed after the event.
  • In two of the five cases, the completed evaluation grids were not on file.
  • In one of the five cases, the Committee's evaluation report had been prepared after the event. In fact, the Evaluation Committee had met after the event to determine whether or not to award the grant.

The Risk-Based Audit Framework (RBAF) has not been reviewed by March 2006 (due date indicated in the action plan). This action has been postponed to September 2006. It should be noted that at the time of the audit in January 2005, we reviewed all the risks identified in the RBAF along with the manager responsible for the G&C Program and it took an hour or so to do the review. We asked management that the same be done without the Audit, Evaluation and Review Directorate on hand. The delay in carrying out this action is not justified.

With respect to the performance measurement required under the RMAF, no action has been taken, despite the March 2006 due date.

Canadian Astronaut Office (CAO)

Audit project objective

The purpose of the audit project was to assess the degree to which current systems, procedures and resources relative to all aspects of managing the Grant and Contribution Program to Support Awareness, Research and Training in Space Science and Technology are efficient, effective and economical.

The Risk-Based Audit Framework (RBAF) of this G&C Program states that an audit of the program, including of both the grant and contribution components, as well as a review of the risk management plan will be conducted in the second year of the program.

Recommendations

In June 2005, we reported that the Canadian Astronaut Office (CAO) generally kept administrative files to ensure a satisfactory audit trail, as required under the TBS Policy on Transfer Payments. However, we noted that improvements could have been made to procedures pertaining to the verification of the eligibility of recipients and to the commitment of funds.

Furthermore, we noted that, despite the fact that most of the risks identified in the RBAF had not been systematically reviewed, appropriate mitigation measures had been implemented for most of the risks identified. With regard to the RMAF, we noted that no data-gathering system had been implemented to measure program performance.

Implementation status

The Canadian Astronaut Office has carried out all of the elements in its action plan in a highly satisfactory manner. The procedure for verifying the eligibility of recipients was reviewed and implemented for grants awarded in October 2005 and April 2006. Funds are now committed in compliance with the CSA Policy on Commitments.

With respect to the RBAF, the CAO now reviews all of the risks and corresponding mitigation measures on an annual basis.

Since January 2005, a data-gathering system has been implemented to measure program performance with respect to RMAF objectives on an ongoing basis. According to the first results obtained and analysed, the established objectives were met.

External Relations Directorate

Audit project objective

The purpose of the audit project was to assess the degree to which current systems, procedures and resources relative to all aspects of managing the Grant and Contribution Program to Support Awareness, Research and Training in Space Science and Technology are efficient, effective and economical.

The Risk-Based Audit Framework (RBAF) of this G&C Program states that an audit of the program, including both the grant and contribution components, as well as a review of the risk management plan will be conducted in the second year of the program.

Recommendations

In June 2005, we reported that the External Relations Directorate generally kept administrative files to ensure a satisfactory audit trail, as required under the TBS Policy on Transfer Payments. However, we found that the evaluation criteria listed in the evaluation grid for the Support Program for Scientific and Space Conferences were not in line with those approved.

In addition, we noted that the RBAF had not been implemented and that the risks and mitigation measures identified in the RBAF had not been systematically reviewed. With respect to the RMAF, we noted that no data-gathering system had been implemented to measure program performance.

Implementation status

The External Relations Directorate has carried out several actions set out in the action plan. An initial review of the RBAF has been completed and mitigation measures have been implemented. In order to complete the implementation of recommendations related to this component, management must update certain mitigation measures.

Recommendations implemented at less than 50% are related to the implementation of a data-gathering system for the purposes of measuring performance and reviewing longer-term expectations in order to determine whether they are still relevant. These two actions, initially planned for December 2005, are moving along at a satisfactory pace, and the new June 2006 due date is considered reasonable.

With respect to evaluation criteria related to the Conference Program, during an upcoming file review, management will apply the same criteria as those used by the Space Science Branch and ensure that they are in line with those approved.

Corporate Management - Accounting Services

Audit project objective

The purpose of the audit project was to assess the degree to which current systems, procedures and resources relative to all aspects of managing the Grant and Contribution Program to Support Awareness, Research and Training in Space Science and Technology are efficient, effective and economical.

The Risk-Based Audit Framework (RBAF) of this G&C Program states that an audit of the program, including both the grant and contribution components, as well as a review of the risk management plan will be conducted in the second year of the Program.

Recommendations

Policies and Procedures 100%

In June 2005, we reported that Corporate Management - Accounting Services needed to ensure that accounts were verified in compliance with the related policy. During our audit of payment claims, we were unable to confirm whether accounts had been verified in compliance with the policy. At that time, payment claims were not systematically corroborated with supporting documents when such documents were not included with payment claims. In all cases, payments were made even though not adequately supported by justifying documentation.

Implementation status

The only recommendation made was not implemented by the August 2005 due date. The action plan anticipated changes to the CSA Policy on Account Verification and that G&C payment claims would subsequently be verified in accordance with the revised policy. The expected changes to the Policy were as follows:

  • Specify the verification procedures, and
  • Review Appendix A to re-evaluate operation categories to be verified and the related risks.

The Policy has not yet been revised. With regard to the verification of G&C payment claims, based on the information obtained, some clerks responsible for verifying accounts have been thoroughly verifying certain payment claims since January 2006. Still based on information obtained, once the Policy will be revised, this new verification approach should be extended to all payment claims, in accordance with Appendix A of the Policy, which lists transactions subject to verification.

Implementation of the Risk-Based Audit Framework (RBAF)

This part of the report provides an overview of the implementation of recommendations related to the Risk-Based Audit Framework (RBAF) in all directorates/branches involved in managing the various components of the G&C Program.

The RBAF-related recommendations were as follows:

  • Implement the RBAF
  • Regularly review the risks and mitigation measures identified in the RBAF.
  • Ensure that all mitigation measures identified in the RBAF are relevant and modify them, as needed.

The table below indicates that three of the five directorates/branches involved have implemented the recommendations.

As for the directorates/branches that have not implemented these recommendations in full or in part, the time limit set out in the action plan was March 2006 for the Space Science Branch and April 2006 for the External Relations Directorate.

Implementation of the Results-Based Management and Accountability Framework (RMAF)

This part of the report provides an overview of the implementation of recommendations related to the Results-Based Management and Accountability Framework (RMAF) in all directorates/branches involved in managing the various components of the G&C Program.

The RMAF-related recommendations were as follows:

  • Implement a data-gathering system to measure program performance on a regular basis, analyse the results obtained to determine whether they correspond to those expected under the RMAF, and adjust the management framework related to program delivery, as needed.
  • Review longer-term expectations to determine whether they are still relevant and adjust them, as needed.

The table below shows that the Canadian Astronaut Office has fully implemented the recommendations by establishing a survey, data-gathering and results-analysis mechanism, making it possible to assess results under the G&C Program's "Aerospace Medicine Elective Award" component.

With the exception of the Space Science Branch, the other directorates/branches involved in managing the G&C Program have taken steps to implement a data-gathering system to measure program performance.