Language selection


Search veterans.gc.ca

Appendix B.4 – Sampling Plan

Objective

The following sampling plan will be applied in assessing the effectiveness and efficiency of both of the PCP funds' through the application/assessment process and application/payment turnaround times. The sample will also be assessed, to the degree possible, in respect of demonstrated need for the program funds by recipients and progress towards expected program outcomes.

Definition of the population and sampling unit

The assessment will be based on a sample of the total population of program applications between April 1, 2008 and March 31, 2011 identified by program area records. The PCP consists of two separate funds (the Community Engagement Partnership Fund and the Cenotaph/Monument Restoration Program).

Sampling Technique

The sampling methodology to be used will be random sampling. Due to the fact that the overall population is quite small (under 500 for both funds); statistically valid sampling with a high confidence level would require reviewing almost the entire population. After assessing the value of such an endeavour, the evaluation team decided that a random sample would maximize resources and provide a sufficient picture of the population and the attributes to be tested.

The funds have separate applications, assessments, review committees and program management staff; therefore, two samples will be chosen, one from each of the respective fund application populations. In order to provide results that will speak to the total population as much as possible, the two samples will not be weighted or stratified.

Attributes to be tested:

Both program funds:

1. Need

  1. How does requested dollars compare to approved and actual reimbursement?
  2. Is VAC providing support to non-eligible activities? If yes, are there any trends?
  3. Evidence of community support/involvement?
  4. Rationale of review committee for funding decisions?

2. Effectiveness/Efficiency

  1. What is the average turnaround time from application received to payment processed?
  2. Are there any gaps and/or roadblocks identified in processes?
  3. Is the application and approval/denial process efficient?

3. Success

  1. Evidence restoration/event successfully completed?

In addition to the above attributes, the Community Engagement Partnership Fund will also be specifically tested for the following:

4. Need

  1. Do event descriptions include eligible activities?
  2. Are target audiences and nature of the project a fit based on Canada Remembers policy and strategy?
  3. Participant feedback on need for program.

5. Success

  1. Are there planned outcomes of the event?
  2. Are there expected methods of measuring success?
  3. What are the actual outcomes identified?
  4. Are there any unintended impacts identified?

Testing Parameters

Due to the slightly different nature of the two funding programs, different parameters will be used.

Cenotaph/Monument Restoration Program

A sample of 20 percent will be an appropriate sample size to review and provide relevant and reportable information regarding the population. The sample size was decided based on the following: an overall small population (178), a more straight forward funding eligibility, the limited number of staff (2) processing applications, the centralized processing area (Head Office), and the fact that the CMRP has been meeting its target turnaround time 100 percent for the past two years.

Community Engagement Partnership Fund

A sample of 35 percent will be applied to review the CEPF. Though the overall population is small for the CEPF and there are limited staff involved in processing, there are a few key differences between the two funds that have driven the difference in sampling parameters. For example, the processing is more complex as there are various processing options depending on whether the event/initiative is regional versus national and the amount of funds being requested. The eligible criteria and types of funded activities are also more complex. Canada Remembers regional staff is very involved in this fund and provides input to applicants, the application process as well as the decision recommendations. The CEPF's proximity to meeting its standard turnaround time has dropped significantly in the past year.

The following table provides overall details:

Program Population Random Sample
 CMRP  178  36 (20 percent)
 CEPF  443  155 (35 percent)

Sample Size

Based on the above criteria, a total of 191 files will be drawn for review. Additional judgemental sampling may be required to gather sufficient evidence on the effectiveness, efficiency, program need and/or progress towards expected outcomes of the program.

Date modified: