Language selection


2.0 Evaluation Context

This evaluation of the OSI Inpatient Clinic (the RTC) at Ste. Anne's Hospital was conducted in accordance with Treasury Board requirements and guidance material. Based on the Terms of Reference cited in VAC's Multi-Year Risk-Based Evaluation Plan for 2011 - 2016, the evaluation is intended to inform VAC's Deputy Minister and concerned departmental management of the relevance and performance of the RTC. Previous evaluation work by VAC's Audit and Evaluation Division (AED) with respect to operational stress injuries, took place in 2008 when the AED completed an evaluation of the OSI Clinic network. The findings and recommendations of this earlier evaluation have served to inform the study team for the RTC evaluation.

2.1 Evaluation Objectives and Scope

This evaluation is the first assessment of the only federally-owned OSI Inpatient clinic aimed at Veterans, CF and RCMP members. The RTC has been in full operation since mid-February, 2010. Accordingly, the scope of this evaluation has been limited to assessing the relevance of the RTC and only the immediate outcome of the clinic's reach in terms of the RTC's overall performance. While it is premature to assess the intermediate and ultimate outcomes for the clinic, it is noteworthy that there is continuing global research in the area of complex, co-morbid OSIs and on achieving better clinical outcomes. This evaluation will be used to develop policy options and future strategic directions for the RTC. The evaluation meets the Government of Canada requirements under the 2009 Policy on Evaluation, which requires that specific evaluation issues and questions (see Section 2.2 below) be addressed and consideration be given to the risks faced by the RTC.

2.2 Evaluation Issues

The RTC is embedded in the Ste. Anne's Hospital which is cited as a distinct program activity in VAC's 2011 - 2012 Program Activity Architecture. The objectives of the evaluation reflect the core issues cited in Annex A of the Treasury Board's Directive on the Evaluation Function:

  • Issue – Alignment with government priorities: To assess the linkages between the objectives of the Inpatient OSI (RTC) and
    1. federal government priorities and
    2. departmental strategic outcomes.
  • Issue – Consistency with federal roles and responsibilities: To assess the roles and responsibilities for the federal government in administering the Inpatient OSI.
  • Issue – Continued need for a program, policy or initiative: To assess the extent to which the OSI Inpatient Clinic (RTC) addresses a demonstrable need and is responsive to the needs of Veterans, CF and RCMP members.
  • Issue – Achievement of expected outcomes: To assess progress towards expected immediate outcomes with reference to program reach.

The first three objectives relate to the core issue of relevance. The last objective relating to performance normally encompasses the assessment of effectiveness, efficiency and economy. Effectiveness is examined by assessing the extent to which the expected outcomes of the RTC are being achieved - including the immediate, intermediate and ultimate outcomes. As the RTC has only been in full operation since mid-February 2010, the evaluation is limited to assessing the immediate outcome related to program reach. The efficiency and economy of the RTC is also a scope limitation.

2.3 Evaluation Design

This evaluation employs multiple lines of evidence, both qualitative and quantitative in nature. The lines of evidence were accrued from a literature review, including research studies, and interviews with key informants and subject matter experts, the RTC and NCOSI professional staff as well as VAC regional and district offices, DND senior mental health-related staff and various Operational Stress Injury Social Support (OSISS) related peer support individuals. This evidence was augmented with research and clinical data as well as the commentary of subject matter experts in the United States and Australia to derive the evaluation findings and form the evaluation team's recommendations. The findings address the core issues cited above and will assist senior departmental management in making future decisions regarding the design and delivery of inpatient services for Veterans, CF and RCMP members.

The external analyses in support of this evaluation can be found in Annex C.

Date modified: