Language selection

Post Payment Verification - Assurance Audit - August 2008

ACKNOWLEDGEMENT

The Audit and Evaluation Division would like to acknowledge all those who assisted us with this audit. In particular, the audit team acknowledges the cooperation and assistance given to us from the Corporate Internal Control Section of Finance Division.

Executive Summary

The Post Payment Verification Assurance Audit was approved by the Veterans Affairs Canada (VAC) Audit and Evaluation Committee on March 21, 2006. The purpose of this audit is to provide assurance that VAC's management frameworks and practices regarding post payment procedures are compliant with Treasury Board Account Verification Policy, Financial Administration Act (FAA) and the Payments and Settlements Requisitioning Regulations, 1997.

VAC's post payment procedures were assessed for efficiency, effectiveness and adequacy of information used for decision-making. This report outlines observations and findings from the review of VAC's Quality Assurance Framework of the Account Verification Process. Contained within this report are ten recommendations relating to the Department's need to clarify its definition of critical errors, improve the accuracy and consistency of its post payment verification reviews, implement a follow-up process for identified errors, provide additional training for staff and develop a quality assurance process.

This audit was conducted in accordance with Standards for the Professional Practice of Internal Auditing adopted by the Government of Canada. These standards require that the audit be planned and performed to obtain reasonable assurance that the post payment verification process is in accordance with governing authorities. Based on the findings, the audit team cannot provide this assurance as, in some cases, it was determined that VAC's management frameworks and practices regarding post payment verification were not in compliance with governing authorities.

Recommendations:

  • R1 It is recommended that the Director General, Finance Division update the Financial Policy and Procedures Manual Chapter 5 Policy on Account Verification to better reflect the requirements of the TB Account Verification Policy.
  • R2 It is recommended that the Chief, Corporate Internal Control Section update VAC's Quality Assurance Framework of the Account Verification Process to better reflect the requirements of the TB Account Verification Policy.
  • R3 It is recommended that the Assistant Deputy Minister, Corporate Services, in collaboration with the Assistant Deputy Minister, Service Delivery and Commemoration, ensure that post payment verification reviews are completed on a timely basis.
  • R4 It is recommended that the Director General, Finance Division update the policies and procedures for conducting post payment verification reviews and communicate them with applicable staff.
  • R5 It is recommended that the Chief, Corporate Internal Control Section modify VAC's Quality Assurance Framework of the Account Verification Process, the related review procedures and reports to incorporate a standardized classification of critical errors with clear definitions.
  • R6 It is recommended that the Senior Assistant Deputy Minister of Policy, Programs and Partnerships and the Assistant Deputy Minister of Service Delivery and Commemoration, in consultation with Finance Division, update the Veterans Programs and Policy Manual and other business support tools to address identified gaps relating to FAA policies and regulations.
  • R7 It is recommended that the Director General, Finance Division develop and implement training on post payment verification review procedures for the Financial Review and Control Units.
  • R8 It is recommended that the Director General, Finance Division develop a quality assurance function for the post payment reviews conducted by the Financial Review and Control Units.
  • R9 It is recommended that the Chief, Corporate Internal Control Section share the results of VAC's Quality Assurance Framework of the Account Verification Process with financial officers who sign pursuant to Section 33 of the FAA.
  • R10 It is recommended that the Director General, Finance Division in consultation with the Director General, National Operations Division expand VAC's Quality Assurance Framework of the Account Verification Process to include a follow-up process for critical errors and observations noted in the reviews.

1.0 Background

Departmental account verification processes are designed and operated in a way that maintains probity while taking into consideration the varying degrees of risk associated with each payment. Treasury Board's (TB's) account verification policy states that financial officers with payment authority pursuant to the Financial Administration Act (FAA) Section 33, must provide assurance of the adequacy of the Section 34 account verification, and be in a position to state that the process is in place and is being properly and conscientiously followed.

High risk transactions must be subjected to a review of all relevant aspects of the transaction. For low and medium risk transactions, it is normally sufficient to select a sample of the transactions and review only the most relevant aspects of each selected transaction. Departments must ensure that the sampling methodology is consistent with sound sampling theory and practice.

The preferred approach to sampling is statistical sampling. Where statistical sampling is not implemented, departments must put into practice an alternative process that will give adequate assurance as to the effectiveness of the account verification process performed under FAA Section 34.

The account verification process for VAC was developed by the Corporate Internal Control Section of the Finance Division and is reviewed yearly by the Senior Financial Officer of the Department.

In February 2005, a review conducted by the Office of the Auditor General of Canada concluded that VAC was not properly certifying payments pursuant to Section 33 of the FAA. Audit work, previously conducted by the Audit and Evaluation Division (AED), also identified weaknesses with Section 33 and Section 34 processes within VAC.

2.0 Objectives

The objectives of the audit were to:

  • Assure VAC has appropriate management frameworks in place to allow for compliance with governing authorities.
  • Assure VAC is compliant with governing authorities relating to post payment assessment of the account verification process.
  • Assess risk management practices which identify and distinguish the risk levels associated with transactions displaying various risk characteristics.
  • Determine if the post payment process is efficient and effective and carried out in a timely manner.
  • Assess the adequacy of information used for decision-making and reporting.
  • Identify potential opportunities to modify the post payment verification process.

3.0 Scope

The scope of the audit included all aspects of the account verification process which should be in place and functioning effectively to ensure that VAC complies with policy, procedural and monitoring requirements of the Treasury Board Policy on Account Verification. The effectiveness and consistency in application of the Quality Assurance Framework within Head Office, Finance Division and the regional finance offices were examined for the period April 1, 2005 to March 31, 2006. The audit focussed on the post payment verification process within the Department and is not intended to provide an independent assessment of VAC's level of compliance to FAA Section 34.

4.0 Methodology

The audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada. These standards require that the audit be planned and performed to obtain reasonable assurance that VAC is following governing authorities for the account verification process.

Audit fieldwork was primarily carried out in VAC Head Office between March 2007 and June 2007. Some interviews were held in the Ontario Regional Office and the Western Regional Office. Specific audit activities included the following:

  • review of applicable government policy and legislation;
  • review of VAC's Quality Assurance Framework of the Account Verification process and of VAC's Financial Policy and Procedures Manual (FPPM) associated with account verification;
  • random stratified sampling of post payment reviews was tested for compliance to review procedures;
  • interviews with management and staff of VAC's account verification areas; and
  • contact with other federal departments to learn their account verification process.

5.0 Observations and Recommendations

5.1 Management Framework for Post Payment Verification

VAC's Financial Policy and Procedures Manual (FPPM), outlines the Department's policy and procedures for account verification. Financial Policy, Planning and Systems Directorate, at Head Office, is responsible for the content of the FPPM. Employees with financial responsibilities are required to familiarize themselves with those sections of the FPPM that are applicable to their areas of responsibility and to comply with them.

The detailed post payment procedures for the Department are contained within VAC's Quality Assurance Framework. The Corporate Internal Control Section, Finance Division has primary control for VAC's Quality Assurance Framework of the Account Verification Process which provides appropriate segregation of duties between the quality assurance review function and Section 34 line managers. This document details the sampling model used for selecting transactions to assess the account verification process pursuant to Section 34 of the FAA for the Department's multiple payment streams.

These documents were compared to the requirements within the Account Verification Policy, FAA, and Payments and Settlements Requisitioning Regulations, 1997 to verify if the Department's frameworks and policies are complying with governing authorities. The objective of Treasury Board's Account Verification Policy is to ensure that accounts for payment and settlement are verified in a cost-effective and efficient manner while maintaining the required level of control. The policy prescribes procedural requirements for the account verification for FAA Section 34, and for the quality assurance of the adequacy of Section 34 account verification.

Observations

The FPPM Part 2 - Chapter 5, Policy of Account Verification describes the policy to a minimal extent and outlines procedures for account verification of goods and services and grants and contributions. The FPPM policy does not clearly document risk considerations as required by Section 5.1.A of the TB Account Verification Policy. As well, TB Account Verification Policy requires risk levels to be identified if the department intends to implement variations to the extent of verification, which VAC had not identified.

TB Policy allows for account verification to be completed after payment has been made provided the claim meets certain criteria. The Department has implemented this TB policy on transactions such as telephone bills. The FPPM Policy of Account Verification has not been expanded to include this practice.

Specific policies and procedures for staff to follow for the verification of accounts are not available for the various payment streams. These policies and procedures should take into account the Department's environment, including the level of decentralization and automated expenditure management systems in place. The roles and responsibilities of each officer should be clearly documented and communicated.

VAC's Quality Assurance Framework of the Account Verification Process, for the most part, outlines the quality assurance process for the adequacy of Section 34 account verification, including transaction profiling, roles and responsibilities and sampling framework. However, there are four requirements of the TB Account Verification Policy which are not included in VAC's Quality Assurance Framework of the Account Verification Process. The first is the requirement that the account verification process be considered as part of the Department's annual risk based audit planning. Secondly, although it has been the practice for the Senior Financial Officer to approve the sampling plan and periodic updates of the plan, this requirement is not clearly stated in VAC's Quality Assurance Framework of the Account Verification Process. The third requirement which was not included in VAC's Quality Assurance Framework is that Section 33 officers are not accountable for account verification errors before payment requisitioning for those payment streams where sound statistical sampling is implemented. This clarification of the responsibilities of the Section 33 officer is an important part of the account verification process. Finally, the TB Account Verification Policy and VAC's Quality Assurance Framework of the Account Verification Process do not allow for withdrawal of Section 34 authority in cases where critical error rates are consistently above acceptable levels.

The FPPM and VAC's Quality Assurance Framework of the Account Verification Process adequately encompass the account verification processes and procedures relating to crown debts and payments subject to Powers of Attorney as required by the TB Account Verification Policy.

R1 Management Response

  • R1 It is recommended that the Director General, Finance Division update the Financial Policy and Procedures Manual Chapter 5, Policy on Account Verification, to better reflect the requirements of the TB Account Verification Policy.

Management agrees with this recommendation. The Financial Policy and Procedures Manual Chapter 5 Policy on Account Verification will be updated to better reflect the requirements of the TB Account Verification Policy.

Management Action Plan
Corrective Action to be taken Office of Primary Interest Target Date
1.1 The Financial Policy and Procedures Manual Chapter 5 Policy on Account Verification will be updated to better reflect the requirements of the TB Account Verification Policy. Financial Policy and Corporate Internal Control December 2008

R2 Management Response

  • R2 It is recommended that the Chief, Corporate Internal Control Section update VAC's Quality Assurance Framework of the Account Verification Process to better reflect the requirements of the TB Account Verification Policy.

Management agrees with this recommendation. A review of the Quality Assurance Framework of the Account Verification Process will be undertaken and updated to better reflect the requirements of the TB Account Verification Policy.

Management Action Plan
Corrective Action to be taken Office of Primary Interest Target Date
2.1 The Quality Assurance Framework of the Account Verification Process has been updated to better reflect the requirements of the TB Account Verification Policy. Corporate Internal Control Completed

5.2 Compliance with VAC's Quality Assurance Framework

The objective of VAC's Quality Assurance Framework of the Account Verification Process is "to provide the framework for assessing of the account verification process pursuant to Section 34 of the (FAA) for the Department's multiple payment streams." The framework outlines the quality assurance process, assignment of responsibilities and sampling framework for post payment verification.

Observations

VAC's Quality Assurance Framework of the Account Verification Process requires the Financial Review and Control Units, in Head Office and regional offices, to carry out quality control review of transactions within four weeks after receipt of the monthly sample. The reviews of transactions are not being completed on a timely basis. Therefore, the results of VAC's Quality Assurance Framework of the Account Verification Process, required to be reported annually, are not occurring as required.

In discussions with the Corporate Internal Control Section, the results of VAC's Quality Assurance Framework of the Account Verification Process for 2005-2006 were expected to be completed by August 2007. The Financial Review and Control Units in the regions and Head Office are currently more up to date with their reviews which will enable the 2006-2007 results to be released in the fall of 2007. VAC's Quality Assurance Framework of the Account Verification Process for the fiscal year beginning April 2007 changes the time frame of the verification process to a calendar year. This will provide more time for the Financial Review and Control Units in the regions to complete their work and submit it to the Corporate Internal Control Section. The results of VAC's Quality Assurance Framework of the Account Verification Process are expected to be completed and ready for the annual Office of the Auditor General's audit of Public Accounts.

The assignment of responsibilities within VAC's Quality Assurance Framework of the Account Verification Process states that Corporate Internal Control Section is to develop the verification sampling plans and procedures for all payment processes and communicate the policy and procedures for staff to follow. While the sampling plans are developed and updated as necessary, the procedures are outdated and not available for all payment streams. There are no procedures for the Treatment Benefits Program or for Disability Pensions. Procedures for other payment streams were written primarily in 2003. Corporate Internal Control Section has noted this deficiency and is already working to update the policy and procedures.

R3 Management Response

  • R3 It is recommended that the Assistant Deputy Minister, Corporate Services, in collaboration with the Assistant Deputy Minister, Service Delivery and Commemoration, ensure that post payment verification reviews are completed on a timely basis.

Management agrees with this recommendation. Financial Review and Control staff are working to complete overdue post payment verification reviews. In addition, resources will put in place to ensure that post payment verification reviews are completed on a timely basis going forward.

Management Action Plan
Corrective Action to be taken Office of Primary Interest Target Date
3.1 Identify regional resource requirements required to carry out the post payment verification reviews on a timely basis. ADM, Corporate Services September 2008
3.2 Ensure that adequate regional resources are in place to carry out these reviews. ADM, Service Delivery and Commemoration April 2009
3.3 The post payment verification reviews for 2008 will be completed prior to the Auditor General Public Accounts Audit in June 2009. Overtime may be required to complete these reviews by the target date. ADM, Corporate Services / ADM, Service Delivery and Commemoration May 2009

R4 Management Response

  • R4 It is recommended that the Director General, Finance Division update the policies and procedures for conducting post payment verification reviews and communicate them with applicable staff.

Management agrees with this recommendation. The post payment verification procedures will be updated and disseminated to applicable staff.

Management Action Plan
Corrective Action to be taken Office of Primary Interest Target Date
4.1 The review procedures have been updated for VIP and WVA. Corporate Internal Control Completed
4.2 The procedures for Health Care Travel and O & M are currently under review. Corporate Internal Control December 2008

5.3 Risk Management Practices

TB policy states that the quality assurance processes used to assess the adequacy of the account verification system must be tailored to reflect the risk level of the transactions under review. In profiling the transactions, the risks associated with each payment category are evaluated with criteria such as sensitivity and susceptibility of error in the payment process. Sensitivity is based on the type of payment, the complexity of the policies to consider and/or the dollar value. The susceptibility to error in the payment process is based on the level of confidence in the existing account verification process. Factors considered include prior error rate experience, competence, volume and complexity of transactions, and the degree of automation in the payment process. The result of the profiling or risk determination process is that all departmental transactions are categorized into four categories, high risk where 100% pre payment verification is required, high risk where 100% post payment verification is required, medium risk using post payment sampling or low risk using post payment sampling (see Annex B).

Observations

The TB Policy requires that the sampling methodology used be consistent with sound sampling theory. Within VAC, appropriate risk management practices for sampling which identify and distinguish the risk levels associated with transactions displaying various risk characteristics have been established and implemented. The population of financial transactions includes the entire scope of Departmental transactions excluding payments for salary and benefits. The described risk assessment of financial transactions has been established and implemented for some time by Corporate Internal Control Section. Each year, as appropriate, the sampling attributes are adjusted depending on the susceptibility of error factors.

5.4 Critical Errors

VAC's Quality Assurance Framework of the Account Verification Process and the TB Account Verification Policy are based on statistical sampling of financial transactions, post payment, to assess the account verification process pursuant to Section 34 of the FAA. The sampling methodology used is referred to as attribute sampling. Attribute sampling is directed toward estimating the proportion of a payment stream that either has or does not have a particular attribute. Attribute sampling is concerned with rates of occurrence and therefore, from a financial officer's point of view, with error rates.

According to VAC's Quality Assurance Framework of the Account Verification Process, an error is anything that would be serious enough to require that payment be held up or cancelled until correction had been made, if it was found pre payment. Errors are classified in the following categories:

  1. Incorrect Payment - incorrect payee name or an incorrect amount paid.
  2. Improper Payment - an unauthorized payment, an appropriation error, a violation of a statute, lack of authority or noncompliance to central agency directives.

Observations

According to the TB Account Verification Policy, financial officers with payment authority pursuant to FAA Section 33 must provide assurance of the adequacy of the Section 34 account verification and be in a position to state that the process is in place and is being properly and conscientiously followed. The TB Account Verification Policy does not define a critical error or what is considered an acceptable level of critical errors.

Prior to the current TB Account Verification Policy, the effective document was TB Circular 1989-15. It abided by the general rule that critical errors include errors that would be serious enough to require that payment be held until correction has been made. It also divided the critical errors into the same categories as VAC's Quality Assurance Framework of the Account Verification Process as described above.

The audit team reviewed the critical error definition and classification of VAC's Quality Assurance Framework of the Account Verification Process, the review procedures and the reports completed over the years, for the various payment streams. While the general definition of a critical error was constant, the actual classification of critical errors differed over time and between payment streams. For example, as described above, an improper payment is currently classified as: an unauthorized payment, an appropriation error, a violation of a statute lack of authority or noncompliance to central agency directives. These classifications should remain constant and be supported by a clear definition describing each category of an improper payment.

Having a standard definition of critical errors for all payment streams, which is shared with management and the responsible organizations and is clear to the post payment auditors, will contribute to a better report on the assessment of the account verification process pursuant to Section 34 of the FAA. Ultimately, it is management's responsibility to attest to the accuracy of the Department's financial statements.

R5 Management Response

  • R5 It is recommended that the Chief, Corporate Internal Control Section modify VAC's Quality Assurance Framework of the Account Verification Process, the related review procedures and reports to incorporate a standardized classification of critical errors with clear definitions.

Management agrees with this recommendation. A standardized classification of critical errors will be issued with the updated policies and procedures.

Management Action Plan
Corrective Action to be taken Office of Primary Interest Target Date
5.1 A standardized classification of critical errors has been added to the Quality Assurance Framework of the Account Verification Process . These are reflected in the review procedures as they are being updated. Corporate Internal Control December 2008

5.5 Quality of the Post Payment Verification Reviews

To provide an opinion on the quality of the post payment verification reviews, the audit team used two approaches. First, the policies and procedures were reviewed to determine if they were relevant. Secondly, a random sample of the completed post payment reviews was assessed against existing policies and procedures produced by Corporate Internal Control Section, to determine if the policies and procedures were being applied consistently.

The post payment verification reviews were conducted in Head Office and in four regions following the areas of responsibility outlined in VAC's Quality Assurance Framework of the Account Verification Process. During 2005 - 2006, the regions completed reviews for operating expenditures, Veterans Independence Program (VIP), Veteran's health related travel, and War Veterans Allowance (WVA), with one noted exception. The Quebec Region was not required to complete post payment verification reviews for operating expenditures as pre payment verification was occurring.

As illustrated in the table below, a random sample of files stratified by region was drawn from the post payment verification reviews completed for operating expenditures, VIP and Veteran's health related travel. Excluded from the sample were Disability Pension and Treatment Benefit transactions, as they were completed by the Financial Review and Control Units in Head Office, and WVA transactions which were determined to be a low risk based on the results of the post payment verifications.

Number of Post Payment Verification Reviews
Region Operating Expenditures VIP Veterans Health Related Travel Total Reviews Audit Sample Size
Atlantic Region 62 420 103 585 184
Quebec Region 0 402 42 444 184
Ontario Region 152 453 98 703 184
Western Region 120 456 124 700 184

Observations

VAC's Quality Assurance Framework of the Account Verification Process assigns Corporate Internal Control Section the responsibility to develop the verification sampling plans and procedures for all payment processes and to communicate the policy and procedures for Financial Review and Control Units to follow under this plan. Such procedures exist for all payment streams with the exception of Disability Pensions and Treatment Benefits Program.

The existing procedures were written in 2003 and have not been updated in accordance with program changes. For example, significant changes to the Regulations for VIP have occurred since the procedures were written in 2003 which are not reflected in the review procedures. There have also been changes to the TB Commitment Policy which are not reflected in the review procedures for operating and maintenance expenditures.

In some cases, the requirements of the Veterans Programs and Policy Manual (VPPM) and other business support tools conflicted with the requirements of the Quality Assurance Framework's review procedures. The VPPM and other business support tools are used by VAC employees to conduct their work and implement the programs of the Department. Therefore, these tools need to encompass the requirements and guidelines set out in the various legislation and regulations for the individual programs as well as encompass the requirements of legislation, regulations and TB policies for how government business is to be conducted. The review procedures for Veteran's health related travel, for example, have a requirement to confirm that the client attended a medical appointment. However, the requirement to get confirmation of a medical appointment is not within the VPPM. The procedures followed would have met the requirements of the VPPM but would still be noted as a error during the post payment verification review.

Furthermore, interviews with regional staff responsible for conducting the post payment verification reviews indicated that they were not aware of review procedures produced by Corporate Internal Control Section. As well, the review sheets were adjusted by regional staff in order to suit their own preferences. This reduces the consistency of the post payment verification reviews.

At the time of this audit, the Corporate Internal Control Section was working on updating and creating procedures for all payment streams. As part of this process, it would be beneficial if the new review procedures are accompanied by training to help ensure that the post payment verification reviews are being completed accurately and consistently.

The results of the audit team's random sampling of completed post payment verification reviews showed that the review procedures for VIP, Veteran's health related travel and operating maintenance were not being consistently followed. A significant number of reviews sampled did not have the supporting information attached, as required by the review procedures. The most common types of missing information were care plans, contribution arrangements, financial coding or expenditure initiation. The absence of these types of supporting information in the review files brings into question if the post payment reviewer properly verified the Section 34, as there was no evidence on the review file to support that all required information was reviewed.

In addition, the Corporate Internal Control Section does not conduct a quality check of the post payment verification reviews completed in the various regions and Head Office. A quality review process would enhance the accuracy and reliability of the work being conducted.

R6 Management Response

  • R6 It is recommended that the Senior Assistant Deputy Minister of Policy, Programs and Partnerships and the Assistant Deputy Minister of Service Delivery and Commemoration, in consultation with Finance Division, update the Veterans Programs and Policy Manual and other business support tools to address identified gaps relating to FAA policies and regulations.

Management agrees with this recommendation. The VPPM and business support tools will be updated to address any identified gaps relating to FAA policies and regulations. In addition, action is underway to identify and address the most critical benefit grid issues.

Management Action Plan
Corrective Action to be taken Office of Primary Interest Target Date
6.1 Develop a mechanism to ensure that all future VPPM or business support tool updates include consultation with Finance Division. Senior ADM, Policy, Programs and Partnerships / ADM, Service Delivery and Commemoration September 2008
6.2 Revise the VPPM and other business support tools to address gaps relating to FAA policies & regulations that have already been identified. Senior ADM, Policy, Programs and Partnerships / ADM, Service Delivery and Commemoration December 2008
6.3 Identify and address additional priority gaps in the VPPM or other business support tools. Senior ADM, Policy, Programs and Partnerships / ADM, Service Delivery and Commemoration March 2009

R7 Management Response

  • R7 It is recommended that the Director General, Finance Division develop and implement training on post payment verification review procedures for the Financial Review and Control Units.
Management Action Plan
Corrective Action to be taken Office of Primary Interest Target Date
7.1 Finance Division will develop financial management training for VAC personnel. The training will include a component on the completion of PPV reviews. January 2009 will see the development of a plan for the training. July 2009 will see the launch of this training. Corporate Internal Control January 2009

R8 Management Response

  • R8 It is recommended that the Director General, Finance Division develop a quality assurance function for the post payment reviews conducted by the Financial Review and Control Units.

Management agrees with this recommendation. A quality assurance function has been developed to assess the post payment reviews conducted by the Financial Review and Control Units.

Management Action Plan
Corrective Action to be taken Office of Primary Interest Target Date
8.1 The Regional Financial Review and Control units now submit the back-up documentation with their Review results. An officer in the Corporate Internal Control unit verifies their work against the established post-payment Review procedures for every transaction in the sample. Corporate Internal Control Completed

5.6 Information Used for Decision-making

The purpose of VAC's Quality Assurance Framework of the Account Verification Process is to provide a method for assessing the account verification process pursuant to Section 34 of the FAA, to ensure the Department is compliant with Treasury Board Regulations and Policies, and to ensure adequate control procedures are in place and functioning appropriately.

Observations

The results of VAC's Account Verification Process Reports for the last number of years have been late, incomplete, and, for the most part, reporting the same issues year after year. The results are shared with the responsible organizations originating the transactions, the Senior Financial Officer, the Director General, Finance Division, the Director General, Audit and Evaluation, and the Office of the Auditor General of Canada.

Based on audit interviews conducted, the results of VAC's Quality Assurance Framework of the Account Verification Process are not copied to Section 33 officers. The TB Account Verification Policy requires Section 33 officers review post payment results to contribute to their ability to provide assurance of the adequacy of the section 34 account verification. Treasury Board Account Verification Policy identifies that responsibility for the system of account verification and related financial control rests with those officers who are delegated payment authority pursuant to FAA Section 33. The Policy also states that financial officers with payment authority, (FAA Section 33), must provide assurance of the adequacy of the FAA Section 34 account verification and be in a position to state that the process is in place and is being properly and conscientiously followed prior to exercising their payment authority.

With the exception of Disability Pensions, critical error rates for the past number of years have consistently exceeded the maximum tolerable error rate stated in VAC's Quality Assurance Framework of the Account Verification Process. While some of the reports contained recommendations to address the critical errors, there is no follow-up process in place to ensure that the critical errors are addressed. This follow-up process will help ensure that critical errors are dealt with in a timely fashion.

R9 Management Response

  • R9 It is recommended that the Chief, Corporate Internal Control Section share the results of VAC's Quality Assurance Framework of the Account Verification Process with financial officers who sign pursuant to Section 33 of the FAA.

Management agrees with this recommendation. The results will be shared with the Regional Directors Finance for distribution to all financial officers who sign pursuant to Section 33 FAA

Management Action Plan
Corrective Action to be taken Office of Primary Interest Target Date
9.1 Effective with the 2007 reviews, Corporate Internal Control will share the results of VAC's Quality Assurance Framework of the Account Verification Process with the Regional Directors Finance for distribution to all financial officers who sign pursuant to Section 33 FAA.. Corporate Internal Control August 2008

R10 Management Response

  • R10 It is recommended that the Director General, Finance Division in consultation with the Director General, National Operations Division expand VAC's Quality Assurance Framework of the Account Verification Process to include a follow-up process for critical errors and observations noted in the reviews.

Management agrees with this recommendation. A process will be established with the National Operations Division to expand VAC's Quality Assurance Framework of the Account Verification Process to include a follow-up process for critical errors and observations noted in the reviews.

Management Action Plan
Corrective Action to be taken Office of Primary Interest Target Date
10.1 In March 2008, a committee consisting of Associate DG Finance, Chief, Corporate Internal Control and Director, Program Management and Operational Guidance, along with other Program managers on an as-needed basis, was formed. This group meets monthly to ensure appropriate action is taken to address critical errors and observations noted in the PPV reviews. Corporate Internal Control Completed
10.2 Another committee, consisting of the Chief Corporate Internal Control and specialist program managers was established in late 2007 to implement corrective action addressing specific critical errors. This group meets, on a minimum, on a monthly basis. Corporate Internal Control Completed

6.0 Distribution

  • Deputy Minister
  • Chief of Staff to the Minister
  • Chair, Veterans Review and Appeal Board
  • Senior Assistant Deputy Minister, Policy, Programs and Partnerships
  • Assistant Deputy Minister, Service Delivery and Commemoration
  • Assistant Deputy Minister, Corporate Services
  • Director General, Finance Division
  • Director General, National Operations Division
  • Chief, Corporate Internal Control Section
  • Deputy Coordinator, Access to Information and Privacy
  • Program Analyst, Treasury Board of Canada, Secretariat
  • Office of the Auditor General
  • Comptrollership Branch

Annex A - Sampling Methodology

A random sampling methodology of post payment verification reviews was applied selecting from the entire population of claims from fiscal year 2005-2006. The sample size was determined according to the level of confidence required in the same proportion as the number of clients in the database. The following criterion determines required sample size:

  • Confidence Level - The degree of certainty with which the error rate in the sample matches the error rate of the population. The higher the level of confidence, the larger the sample size required.
  • Maximum Tolerable Error Rate (MTER) - The maximum acceptable occurrence of critical errors. If a significant variance exists between the resulting error rate and MTER, the sampling parameters would be adjusted with a view to establishing which area(s) is causing the errors.
  • Margin of Error or Precision Level - Is a measure of the possible difference between the resulting estimated error rate and the true error rate in the total population.

Population Size - Total number of post payment verification reviews by region
  Atlantic Quebec Ontario Western
Population:
Confidence Level: 95% 95% 95% 95%
Maximum Tolerable Error Rate: 5% 5% 5% 5%
Margin of Error: +/-2% +/-2% +/-2% +/-2%

Annex B - Risk Matrix

The VAC Quality Assurance Framework of the Account Verification Process profiles transactions, by risk associated with each payment category. The criteria for evaluation include sensitivity and susceptibility to error in the payment process.

Risk Matrix
Susceptibility to Error Transaction Type
  Non-Sensitive Sensitive Highly Sensitive
Minor
(error rate ≤ 4%)
Low Risk
Post Payment
Sampling
Low Risk
Post Payment
Sampling
High Risk
100% Post Payment
Verification
Moderate
(error rate 4 to 10%)
Low Risk
Post Payment
Sampling
Low Risk
Post Payment
Sampling
High Risk
100% Post Payment
Verification
Severe
(error rate > 10%)
Medium Risk
Post Payment
Sampling
High Risk
Post Payment
Sampling
100% Pre payment
Verification
Required
Date modified: