The overarching evaluation assesses Commemoration from a strategic perspective, providing information on the linkages between programs, and capturing items that the previous individual evaluations (Evaluation of Commemorative Benefits and Services and Evaluation of Public Recognition and Awareness) were not able to fully assess and report on.
The evaluation was conducted in accordance with the directive and standards specified in Treasury Board of Canada’s 2016 Policy on Results. It covers the time frame from April 1, 2012 to September 30, 2017, and was conducted between September 2017 and December 2017.
| Planning and Funding |
|---|
| Long-term strategic planning |
| Program funding |
| Funding pressures |
| Commemorative Integrity |
| Canadian National Vimy Memorial, Beaumont-Hamel Newfoundland Memorial |
| Alignment |
| Opportunities for enhanced alignment internally - Departmental |
| Opportunities for enhanced alignment internally - Divisional |
| Management/Organizational Structure |
| Do the current structures facilitate efficient and effective decision making for Commemorative objectives/priorities? |
| Opportunities |
| International comparison |
| Best practices |
2.1 Methodology
The research methodology incorporated multiple lines of evidence, ensuring reliability of collected information and reported results. The evaluation was formativeFootnote 2 in nature and relied on a mix of qualitative and quantitative lines of evidence, including evidence gathered in previous commemorative evaluations, as well as the Audit of Overseas Ceremonies and Events, as shown in Table 2.
| Methodology | Source |
|---|---|
| Non-departmental document reviews | Key non-departmental documents such as: Parliamentary reports, Federal Budget documents, Speeches from the Throne, media articles, and evaluations of commemoration programs in other federal departments. |
| Interviews and/or work observation | Approximately 20 interviews with key informants were conducted during the fieldwork phase of this evaluation. In addition, this report was informed by approximately 40 interviews conducted during the Evaluation of Public Recognition and Awareness and the Evaluation of Commemorative Benefits and Services. Interviews were conducted with VAC senior management; VAC staff involved in the delivery of commemorative programs; VAC partners and external departments delivering similar programs; and other subject matter experts. Direct observations were conducted of VAC memorial sites in Europe. |
| Departmental documentation and secondary research review | The following types of departmental documents/information were reviewed to gain an understanding of the components, their authorities and requirements, and key issue areas: memoranda of understanding and their annexes; previous audits and evaluations; policies; business processes; records of decisions; strategic and business planning documents; performance reports; research papers; survey results; third-party contracts; invoices; social media; media reports; and correspondence. |
| Statistical analysis | Financial and operational data collected by VAC for fiscal years 2012-13 to 2016-17 was examined. |
| Survey | Results from the recent Survey of Commemorative Initiatives conducted during the Evaluation of Public Recognition and Awareness were examined. The survey was completed by 67 VAC staff who self-identified as Canadian Armed Forces (CAF) Veterans or still-serving members. |
2.2 Limitations and Analytical Challenges
The following limitations and challenges were identified:
- The evaluation team did not speak directly with Canadians. Where available, the evaluation team used existing public opinion research.
This limitation/challenge should be considered when reviewing the evaluation findings.