Language selection


Search veterans.gc.ca

4.0 FINDINGS FOR EACH OF THE FIVE OUTCOMES

This section presents the following NVC outcomes: recognition, health, community integration, employment and income. The analysis will focus on the success of achieving these desired outcomes. Each section will provide an overview of the program outcome, effectiveness and efficiency, summary conclusions and recommendations where appropriate.

4.1 Recognition

Outcome

NVC clients feel recognized for their contribution.

Overview

Veterans Affairs Canada exists to “repay the nation’s debt of gratitude toward those whose courageous effects have given us this legacy, and have contributed to our growth as a nation”; part of this responsibility includes recognizing the contribution, achievements, and sacrifices of Canadians during periods of war and peace, including peacekeeping operations.

All VAC programs, through the provision of benefits and services, can arguably, contribute to recognition, including the NVC programs and Canada Remembers (a division of Veterans Affairs Canada dedicated to promoting understanding of Veterans’ sacrifices, engaging communities in remembrance, and keeping the sacrifices and achievement of Veterans alive). Canada Remembers programs and activities include ceremonies and events, cemetery maintenance and memorials and learning initiatives, all of which provide recognition, at a more global level rather than on an individual level. The suite of NVC programs was intended to recognize Veterans for their service to Canada. The Disability Award (DA) Program was designed to provide compensation and is the only NVC program that includes recognition as an outcome.

There was no specific data regarding NVC recognition available to the evaluation team. Therefore, focus groups were conducted to assess opinions and perceptions of the NVC programs including the topic of recognition. The focus group participants stated that VAC recognition is not specific to a program but rather the range of VAC programs and services. Also, the majority of focus group participants indicated that recognition involved treating them with dignity and respect. The 2007 National Client Satisfaction Survey (NCSS) noted that 96 percent of CF Veterans were satisfied with the dignity and respect shown to them by VAC staff.

Focus group participants gave no indication that the Disability Award Program contributes to their sense of recognition, but when asked for additional ways VAC could recognize Veterans, most often participants suggested re-instating the disability pension or increasing the amount of benefits provided. It was even suggested that the introduction of the Disability Award was a way for the Government to divest its responsibility for Veterans. The Disability Award Program was not identified by any focus groups participants as contributing to their sense of recognition.

Additionally, any pertinent information gathered from the Evaluation of Disability Pensions and Awards has been incorporated where appropriate.

Effectiveness

Recognition is a subjective measure, and can be impacted by many factors. This results in challenges when attributing a program’s activities to achievement of program outcomes. As mentioned earlier, the whole array of the Department’s legislation contributes to recognizing Veterans’ contributions. Although the DA program is the only NVC program with recognition as an outcome, the effectiveness in achieving this outcome should be assessed from a broader perspective.

The effectiveness of recognition through the DA Program is diminished due to a number of factors, including the payment delivery method and amount of compensation provided. The DA is a one-time act of recognition and some individuals may no longer feel recognized after they have received their award and the money has been spent. Focus group evidence indicates that individuals don’t feel recognized by the DA Program. However, neither has VAC effectively communicated to CF members, Veterans, families, and staff that the DA Program is intended to provide both compensation and recognition.

The Evaluation of Disability Pensions and Awards noted that VAC does not effectively compensate some seriously disabled Veterans. As a result, recipients may have a reduced sense of recognition. Improved financial benefits to seriously disabled CF members were announced by the Minister of Veterans Affairs in September 2010 to more appropriately compensate and recognize this group.

Efficiency

When discussing other impacts on recognition, timeliness and administrative delays were noted as having a significant negative impact on a recipient’s sense of recognition. The majority of disability award applications require additional information to complete the application process, thereby placing additional stress on recipients and adding to VAC’s administrative workload. This significantly extends the length of time it takes for the applicant to receive a decision on their DA application. Increasing backlogs of applications awaiting processing also adds to the time an applicant must wait for a decision. NVC Evaluation Phase II recommended efficiency improvements to the application process and changes will be made in conjunction with the transformation plan to re-engineer the Disability Benefits Program to reduce complexity and overhaul service delivery.

In 2009-2010, VAC adjudicated 9,219 DA decisions with administrative costs of approximately $1,200 per decision. Of these decisions only 41 percent were completed within the service standard of 24 weeks, which is measured from the time a signed application with the necessary information is received. Additionally, the number of DA applications has been increasing since program inception and has led to a backlog of applications awaiting adjudication. As of March 31, 2010 there were 6,713 DA applications awaiting adjudication. Recently, emphasis has been placed on processing DA applications, and this has reduced the backlog to 3,761 as of September 30, 2010. VAC is taking steps to further reduce the service standard to 16 weeks.

Contrary to recipient concerns, the payment delivery method of one-time payments is an efficient means to compensate and recognize for their service-related disability. Full compensation is received when their condition is stabilized and recipients are free to spend it as they see fit. There is less administrative burden to providing one-time payments compared to previous and other alternatives to administering disability benefits. In response to Veterans’ concerns regarding one-time payments, VAC has recently studied the issue, and the Minister of Veterans Affairs announced alternatives to the
lump-sum payment in November 2010.

Conclusion

  • All of VAC’s programs contribute to recognition.
  • Focus group participants stated that VAC recognition is not specific to a program but rather the range of VAC programs and services.
  • The DA is the only NVC program that includes recognition as an outcome.
  • VAC does not effectively communicate to CF members, Veterans, families, and staff that the DA Program is intended to provide both compensation and recognition.
  • Evidence to date indicates that Veterans don’t feel recognized by the DA Program.

Recomendation

It is recommended that the ADM, Service Delivery, clarify if recognition should be measured under the NVC. Appropriate logic models and measurement tools will need to be developed or revised. (Essential)

Management Response

Management agrees with the recommendation.

In 2006, a logic model was submitted to Treasury Board Secretariat as part of the New Veterans Charter (NVC) Results-Based Management Accountability Framework. This logic model demonstrated how collectively, the re-establishment programs contributed to the overall intent of the New Veterans Charter. Since 2006, logic models and performance measurement plans have been developed for each of the New Veterans Charter programs, demonstrating at a more detailed level, the distinct outcomes of each program.

This process has resulted in two distinct outcomes related to recognition. The first (global) outcome applies to all Canadian Forces Veterans and is intended to reflect how collectively, the suite of NVC programs contribute to modern-day Veterans feeling recognized for their contributions to the safety and security of the country. When this outcome was developed, it was acknowledged that perceived recognition is a subjective experience of each individual Veteran. Additionally it was recognized that numerous other factors contribute to recognition (e.g. Canada Remembers programming including: ceremonies and events; the construction of new and modifications to existing memorials to recognize modern-day Veterans; the development and distribution of learning materials about military achievements and remembrance; partnerships with communities and Veterans' organizations to promote recognition; as well as programs of external organizations and community-based remembrance activities). Given the importance of recognition in the results of the Review of Veterans Care Needs survey, Program Management has measured this outcome for NVC recipients via the VAC Re-establishment Survey since 2007.

The second outcome applies specifically to Canadian Forces Veterans with service related disabilities and is intended to reflect how disability awards contribute to a Veteran feeling that his/her service-related disability has been recognized by the Government of Canada. This outcome was measured via the National Client Survey in 2010.

As is generally the case with outcomes relating to perceptions and attitudes, the global outcome of perceived recognition for one’s contribution to the safety and security of Canada is influenced by a variety of personal and environmental factors. This makes evaluation a challenge and attribution of this outcome to one single NVC programs impossible.

Management Action Plan
Corrective Action(s) to be taken Number OPI (Office of Primary Interest) Target Date
Identify any required amendments to existing program outcomes, amend and communicate as appropriate. Service Delivery Management March 31, 2012

4.2 Health

Outcome

Experience improved health status and functional capacity as a result of access to treatment benefits and rehabilitation services.

Overview

Rehabilitation is a process to restore an individual’s physical, psychological, social and vocational abilities to a level that will allow a Veteran’s to integrate and actively participate in their community. Focus is on the reasonable restoration of an individual’s functioning in five major areas: mental and physical functioning; activities of daily living; family relationships; employment; and community participation. This restoration of functioning is done by working collaboratively with various partners, including the Department of National Defence (DND) and the Service Income Security Insurance Plan (SISIP). SISIP is a long-term disability (LTD) group insurance plan for Canadian Forces (CF) personnel.

Rehabilitation participants with medical and/or psychosocial needs are eligible for treatment benefits which were the basis for their eligibility to the program. The overall goal of these treatment benefits is to improve the participant’s health status and functional capacity in relationship to their identified rehabilitation need. Treatment benefits are provided through Programs of Choice (POC’s) 1 – 14 and details can be found in Appendix G.

Vocational rehabilitation, a component of the Rehabilitation Program, is designed to identify and achieve an appropriate occupational goal for a person with a physical or a mental health problem, given their state of health and the extent of their education, skills and experience. Vocational rehabilitation is analyzed further in Section 4.4 of this report.

Veterans and their families, through the Rehabilitation Program, are eligible for financial support which provides them with income in order to effectively participate in rehabilitation. Group health care insurance is also available to assist Veterans and their families with their transition to civilian life.

To determine if the Rehabilitation Program contributes to improved health status and functional capacity, it is necessary to first understand the baseline condition of individuals when they enter the Rehabilitation Program. Little was previously known about the conditions of Veterans who have been subjected to harsh conditions in combat and disaster relief situations. The File Review, previously described in Section 3.3, is the only source of complete information available describing the health status of individuals at the time of applying. The File Review noted an exceptionally high proportion of individuals reported pain at baseline functioning (83 percent). At least half of the sample self-reported a significant mental health issue (57 percent), a high stress level (51 percent) and presented with a type of operational stress injury (OSI)1 (49 percent). Of the individuals sampled in the File Review, 60 percent had one or more Special Duty Areas2 with Bosnia, Cyprus, Croatia and Afghanistan being the most common areas. As expected, there was an affirmative relationship between having SDA service and a higher incidence of OSIs. However, the File Review noted, an increase in the number of SDAs does not correlate to an increase in the number of OSIs.

LASS: STCL sample was stratified into three groups: individuals participating in New Veterans Charter programs (NVC participants), Veterans in receipt of disability pensions but not participating in NVC programs (DP recipients), and Veterans not participating in VAC programs. According to the survey, individuals participating in VAC programs were worse off for health, disability and determinants of health. NVC participants had the worst status. More than a third who released from service during 1998-2007 did not have an easy adjustment to civilian life.

The Table 2: LASS Comorbidity of Chronic Physical and Mental Health Conditions, clearly demonstrates that 55.2 percent of NVC participants have both a mental and physical health condition compared to 9.8 percent of Veterans who are not VAC participants. This finding supports the long held contention that there is a high degree of co-existent physical and mental health issues present in NVC participants.

As seen in the table, there are large numbers of individuals with health conditions who are not accessing services through the NVC. For example, 91.6 percent of disability pensions and 53.6 percent of Veterans who are not receiving VAC benefits have a physical condition. LASS: STCL noted that individuals not participating in VAC programs were similar to Canadians in the general population in many respects, but on average, had higher rates of some chronic health conditions and disability, and had significant rates of attributing both to military service. These findings suggest unmet needs and/or VAC program reach issues, as noted in the NVC Phase II evaluation report.

Table 2: LASS Comorbidity of Chronic Physical and Mental Health Conditions
% of NVC Veteran participants % of Disability Pension recipients % of Veterans who are not in receipt of VAC benefits
Physical 90.7% 91.6% 53.6%
Mental 59.9% 40.2% 12.8%
Both 55.2% 37.6% 9.8%

Additionally, further corroboration of these findings exist in research conducted by Gordon Asmundson (2000) that concluded "VAC’s CF clients have higher health care utilization and more long term health problems than does a matched comparison group from the general population."

Effectiveness

As of March 31, 2010, 4,448 Veterans applied to the Rehabilitation Program and 4,197 have participated in the program over the four year time span. To date, only 11 percent of participants (441) completed the Rehabilitation Program; however, it is difficult to quantify the success rate of the program, as individuals entered at various points in time since program inception. Individual needs vary from complex to straightforward and, therefore, it is difficult to determine what the completion rate should be. A complete analysis of those who completed and their durations in the program can be found in Appendix H.

The low completion rate may be explained by the complex needs of individuals within the program. Further mitigating factors include:

  • A number of Veterans come into the Rehabilitation Program after completing a two-year program through SISIP LTD which did not lead to successful transition. An in-depth analysis of SISIP participants was not conducted as SISIP was out of scope for the evaluation.
  • Some Veterans require extensive medical and psychosocial services before they can undertake vocational rehabilitation which can easily extend over 24 months.
  • Successful rehabilitation is further complicated by the need to find new employment as opposed to returning to their former employer.

The file review noted that when an individual’s issues were unambiguous, such as a barrier in vocational roles and established resources were available, good outcomes were achieved. On average, the File Review noted that individuals completed the Rehabilitation Program within 20 months. Similarly, according to internal statistical data, participants took an average of 21 months to complete the program.

At the individual level, success varies from those who have completed and have improved health, to those still in the program but who have experienced incremental improvements. Based on the results of the File Review, we can say that 71 percent of participants in the sample have made some degree of overall progress while in the Rehabilitation Program.

The NVC was designed to assist CF members, Veterans and their families to transition into civilian life. The file review noted that participants who showed overall success in the program tended to be married. Focus group participants noted that transition affects not only the Veteran, but the family members as well. To address this need, the NVC was designed to be more family focussed; however, statistical information indicated this shift has not occurred. Analysis of treatment benefits has indicated that over the study period, 158 families of Veterans accessed family benefits totalling approximately $277K over four years. As of March 31, 2010, 16 survivors accessed treatment totalling approximately $50K over the last four years. NVC Phase I evaluation provided an early indication of the disconnect between the expected results for families and the legislative authority within the NVC, which resulted in staff confusion regarding VAC’s role in meeting family needs. Staff are not clear regarding the relationship between families and the NVC suite of programs as evidenced by the number of families who accessed treatment benefits and the gap in outreach activities to families.

Another benefit available to families is group health insurance through the Health Benefits Program. This program is designed to fill gaps in post release health coverage through the Public Service Health Care Plan. As of March 31, 2010 there have been 974 individuals who have accessed health benefits through this program. LASS: STCL information tells us that 95 percent of NVC participants indicate that they have insurance for prescription drugs, as opposed to 89 percent of Veterans who are not in receipt of VAC benefits. Access to health benefits such as, prescription drugs, contributes to improved health.

Additionally, the Financial Benefits Program provides individuals various forms of income-replacement such as Earnings Loss, so that financial obligations can continue to be met while they participate in rehabilitation. As of March 31, 2010 there were 2,089 individuals in receipt of Earnings Loss benefits.

Efficiency

At the time of NVC design, the anticipated program duration for the Rehabilitation Program was to average 24 months. Analysis of the data shows that the length of time an individual is in the Rehabilitation Program can vary considerably. Of the individuals who entered the program in 2006-2007, two thirds are still in the program which indicates that the majority are taking longer than the expected average time to complete.

As shown in Table 3, Duration in the Rehabilitation Program for those still participating, there were 3,351 Veterans (excluding spouses and survivors) with an eligible status as of March 31, 2010. This chart shows that 1,389 (41 percent) have been in the program for 25 months or longer which exceeds the expected average duration.

There are similarities in the analyzed data between the number of Veterans who released from the CF prior to the inception of the NVC (pre 2006) and those who released following (post 2006) program creation and are currently in the program (see Appendix I for a visual representation). One would expect to see fewer Veterans who released post 2006 in the upper duration bands, as the program has only been in existence for four years. It was surprising to note that the numbers of Veterans still in the Rehabilitation Program are nearly evenly split between those who released pre 2006 (53 percent) and those that released post 2006 (47 percent).

Table 3: Duration in the Rehabilitation Program for those still participating
Veterans In Rehabilitation
Length of Time (mths) Release Date pre 2006 Percentage of clients within band Release Date post 2006 Percentage of clients within band Pre and post total In Rehab Percentage of clients within band
< 6 219 12% 274 17% 493 15%
6 - 12 271 15% 381 24% 652 19%
13 - 18 181 10% 257 16% 438 13%
19 - 24 163 9% 216 14% 379 11%
25 - 36 359 20% 280 18% 639 19%
37 - 42 239 13% 106 7% 345 10%
> 42 341 19% 64 4% 405 12%
Total* 1,773 53% 1,578 47% 3,351

*Excluding Survivors and Spouses

In addition to the statistical information presented above, the File Review noted that of those still participating in the Rehabilitation Program, 44 percent require long term support to maximize function or prevent deterioration. Long term support means that it is not anticipated that an individual will meet medical, psychosocial and/or vocational goals in the next six to twelve months. This is based on the observation that the individual has been engaged in the rehabilitation process for an average of 20 months, however, has not achieved many established goals.

Another contributing factor to the long duration in the Rehabilitation Program are those Veterans who have been declared Totally and Permanently Incapacitated (TPI) and continue to have an active rehabilitation plan. TPI means that the Veteran is incapacitated by a permanent physical or mental health issue that prevents them from performing any occupation that would be considered suitable gainful employment.

According to the File Review, approximately 31 percent of the active rehabilitation cases reviewed were for individuals who suffered from a health condition severe enough to designate them as TPI. Staff indicated they were reluctant to designate a Veteran as TPI as this designation has negative connotations and may impair the relationship between the Veteran and the Case Manager. Early in the program there was confusion regarding TPI, additional training was provided and the number of Veterans designated as TPI has increased.

In summary, there are a large number of program participants with significantly complex issues who will require long-term medical and psychosocial benefits to improve and maintain their health status and functional capacity which is beyond the scope of the program as currently defined.

Eligible Veterans with medical and/or psychosocial needs are entitled to treatment appropriate to their particular needs. Individuals are able to access treatment through either their rehabilitation plan or due to their pensioned/awarded condition. Business processes dictate if the treatment provided is directly attributable to the Veteran’s rehabilitation need, the benefit is coded as rehabilitation (R) within the Federal Health Claims Processing System (FHCPS). Benefits related to a Veteran’s pensioned/awarded condition and rehabilitation need are coded as overlap (O). If the benefit is related to the pension/award condition only it is shown in the system as null.

The interpretation of the assignment of these codes varies across the country. In some instances, health benefits that clearly overlap are charged to pension/award. In other instances, codes that are clearly rehabilitation only are charged to either overlap or the Veteran’s pensioned/awarded condition. There is no method to ensure consistency of coding throughout the Department.

Due to the limitations described above, the evaluation team applied logic to available data from the FHCPS such that all treatment benefits received within the Veteran’s rehabilitation plan were considered to be rehabilitation related, which amounted to expenditures of $31.2 M for the 4 fiscal years ending March 31, 2010 (OSI expenditures not included). This is consistent with the outcome that indicates individuals experiencing “improved health status and functional capacity as a result of access to treatment benefits…” therefore, it is reasonable to assume that access to any and all treatment benefits contribute to improved health.

To date, VAC has only been reporting treatment benefits coded as rehabilitation (R). Costs such as drugs, OSI expenditures and portions of health related travel, cannot be appropriately attributed to rehabilitation participants and therefore, are not included in the rehabilitation costs. Expenditures for rehabilitation codes (R) within the participant’s rehabilitation plan dates were approximately $6.8 M as of March 31, 2010. Treatment expenditures for spouses are not captured as they are identified by the Veterans file number only.

An additional analysis of the total treatment costs for Veterans (31.2 M) within the study time period from April 1, 2006 – March 31, 2010 showed 3,451 individuals received treatment benefits within their rehabilitation plan. A total of 16 survivors also received treatment benefits totalling approximately $50K. A survivor is eligible for vocational assistance services if their spouse was a Veteran and died from a service-related injury or disease. They may also be eligible for medical, psychosocial and vocational rehabilitation if these services are necessary for them to benefit from vocational assistance. Figure 1, Breakout of Rehabilitation Program participants, further describes the associated treatment costs for the above noted groups as of March 31, 2010.

Figure 1: Breakout of Rehabilitation Program Participants

Figure 1, Breakout of Rehabilitation Program participants, further describes the associated treatment costs for the above noted groups as of March 31, 2010.

Of the 730, 592 received no treatment at all and 138 received treatment but outside of their Rehabilitation dates

730 no Treatment within their RehabPlan

Total Participating Rehab 4197 Include Veterans, spouse and survivors

3451 veterans receiving treatment totalling $31,271,677

16 survivors receiving treatment within their Rehab plan totalling $ 49,870

Within the 3451, there are 315 TPI clients $6,539,992

Figure 1 can be broken out further by those who are still participating in the program (identified as “In”) and those who have ceased participating in the program (with a status of completed, cancelled or deceased; identified as “Out”).

A breakout of the total Rehabilitation population group into individuals who are “In” and individuals who are “Out” can be found in Figure 2: Breakout of Rehabilitation Program Participants by Ins and Outs, below.

Figure 2: Breakout of Rehabilitation Program Participants by Ins and 0uts

Figure 2: Breakout of Rehabilitation Program Participants by Ins and 0uts

From 4197

IN

Eligible participants 3,424 includes Veterans, spouses and survivors

Within the 2,835, there are 283 TPI Veterans $6,084,626

2,835 Veterans Treatment within their Rehab plan $27,528,777

13 survivors with an eligible status and treatment within their Rehab plan $40,445

OUT

773 out participants (cancelled completed or decreased) includes Veterans, spouses and survivors

616 Veterans Treatment within their Rehab plan $3,741,900

Within the 616, there are 32 TPI Veterans – $455,366

3 out survivors with treatment within their rehab plan – $9,425.00

Of the $31.2 M spent on Veteran treatment benefits as of March 31, 2010 Table 4, Treatment Expenditures for participants by Ins and Outs, shows that $27.5 M (88 percent) of the total treatment expenditures are for individuals who are still participating in the Rehabilitation Program. For those currently participating in the Rehabilitation Program and released prior to 2006, average treatment expenditures are twice as much as for those who released post 2006. Distribution tables for expenditures can be found in Appendix J and costs broken out by POC code can be found in Appendix K.

Table 4: Treatment Expenditures for Participants by Ins and Outs
Veterans
In (Participating) Out (completed, cancelled, deceased)
Pre 2006 expenditures $ 19,783,916 Pre 2006 expenditures $ 2,952,542
Participant count pre 2006 1,611 Participant count pre 2006 396
Average cost pre 2006 per participant $ 12,281 Average cost pre 2006 per participant $ 7,456
Post 2006 expenditures $ 7,745,861 Post 2006 expenditures $ 789,358
Participant count post 2006 1,224 Participant count post 2006 220
Average cost post 2006 per participant $ 6,328 Average cost post 2006 per participant $ 3,588
Total Expenditures for all years $ 27,529,777 Total Expenditures for all years $ 3,741,900
Number of Participants 2,835 Number of Participants 616
Average cost per participant for all years $ 9,711 Average cost per participant for all years $ 6,075
Table 4: Treatment Expenditures for Participants by Ins and Outs
Survivors
In (Participating) Out (completed, cancelled, deceased)
Expenditures $ 40,445 Expenditures $ 9,425
Participant Count 13 Participant Count 3
Average cost per participant $ 3,111 Average cost per participant $ 3,142

Table 4 shows 53 percent of "In" Veterans who released pre 2006 are utilizing 72 percentof total expenditures for the "In" group ($27.5 M). While there is very little difference in the numbers of Veterans who released pre and post 2006, the pre 2006 group requires access to substantially more treatment benefits.

For individuals currently in the program, treatment costs (88 percent) will continue to escalate as program duration cannot be predicted. This finding further demonstrates the requirement for long-term support. When the NVC program was designed, VAC could not predict the need for long-term support, which has resulted in increased costs and is impeding VAC’s ability to achieve the health outcomes established.

There were 592 individuals in the Rehabilitation Program as of March 31, 2010 who did not receive treatment benefits. Of these, 19 were in receipt of Earnings Loss benefits and no other benefit or service.

In addition to treatment benefit costs, there are other direct and indirect costs associated with delivering and supporting the Rehabilitation Program. The input costs being reported and analysed in this evaluation are those that are incurred annually, with related start-up costs being deemed as sunk costs and therefore were excluded from the analysis such as: a task force being set up to oversee the development and implementation of the program, various project structure working groups established under the auspices of the task force, IT costs both in terms of hardware and software development as well as required resources, costs incurred through outside contracts such as the FHCPS, and training for 3,000 employees on the new programs.

It is beyond the scope of this analysis to compare the forecasted and actual sunk costs for the NVC. However, an estimated cost for the Rehabilitation Program was developed based on the treatment cost and the administration costs provided to the evaluation team, amounting to an average weighted cost per individual while in the Rehabilitation Program of $18,322 (excluding sunk costs). The estimated average cost for Earnings Loss was calculated in the same fashion with an average weighted cost per individual of $30,797. Detailed information on the costs can be found in Appendix L.

Conclusions

  • 11 percent (441) of participants have completed the Rehabilitation Program.
  • Based on the File Review of 350 rehabilitation participants, 71 percent have made some degree of overall progress after receiving services.
  • The File Review found 44 percent of participants required long-term support.
  • Veterans requiring long-term maintenance are utilizing the Rehabilitation Program affecting VAC’s ability to achieve program outcomes as designed.
  • A strategy is needed to address the segment of rehabilitation Veterans who require long-term maintenance.
  • 158 families are accessing treatment benefits.
  • The Health Benefits Program fills a gap for the targeted population.
  • Access to income through Earnings Loss allows individuals to focus on rehabilitation.
  • System coding issues and incorrect data entry does not allow for adequate reporting and forecasting.

Recommendation 2

It is recommended that the ADM, Service Delivery lead the development and implementation of a strategy to address the needs of CF Veterans requiring long-term maintenance. (Critical)

Management Response

Management agrees with the recommendation.

Further analysis to determine the precise needs, and the options to meet these needs, will be conducted as part of the Department’s transformation agenda.

Management Action Plan
Corrective Action(s) to be taken OPI (Office of Primary Interest) Target Date
Identify options for meeting the needs of individuals requiring long-term maintenance Rehabilitation Directorate March 2011
Implement changes to address the needs of individuals requiring long-term maintenance through the departmental Transformation Plan Rehabilitation Directorate March 2012

Recommendation 3

It is recommended that the ADM, Service Delivery appropriately revise systems, such that participant progress can be properly determined, and total expenditures for rehabilitation can be accurately allocated and forecasted. (Critical)

Management Response

Management agrees with this recommendation.

Management Action Plan
Corrective Action(s) to be taken OPI (Office of Primary Interest) Target Date
An analysis will be conducted on the assignment of codes and to define the scope of the reporting and systems requirements for the Rehabilitation Program to properly identify treatment and programs costs and to determine individual progress. SDM September 2011
Options based on the findings of the analysis will be examined and developed for management for approval and implementation. SDM December 2011

4.3 Community Integration

Outcome

CF Members, Veterans and their families actively participate or are integrated into their communities

Overview

According to the, Journal of Rehabilitation Research & Development (Resnik, Plow, and Jette, 2009) the ultimate goal of rehabilitative efforts is to help those injured adjust to life at home and in the community, which is also called community reintegration (community integration). Community reintegration (community integration) is especially challenging for injured Veterans because it may be complicated by the co-occurrence of physical injuries with postwar adjustment difficulties, such as post-traumatic stress disorder (PTSD), depression, substance abuse, and severe mental illness. To assist with these challenges, VAC’s Operational Stress Injury Social Support (OSISS) Program provides confidential peer support and social support to CF members, Veterans, and their families, affected by an operational stress injury. Within the NVC suite of programs, the community integration outcome is measured through the Rehabilitation Program.

Research on social relationships and health is based on measures of social ties such as the number of friends or acquaintances, or from perceived social or emotional support, the concept of community belonging measures one’s sense of belonging to the broader community (Ross 2002). Community belonging becomes even more important once members leave the military, since they are leaving their established social networks.

According to the International Classification of Functioning, Disability and Health (ICF), the domain of participation focuses on the person’s involvement in society (i.e. community integration). People are considered to have healthy participation if they take part in all life areas or life situations in which they wish to participate, in a manner or to the extent that is expected of an individual without restrictions in that culture of society.

As an example, a client may present with a knee injury. As a result of this injury, the client has activity limitations such that walking is limited to a block, stair negotiation is painful and standing tolerance is 10 minutes. Depending on the severity of the impairment(s) and the degree of activity limitations and participation restrictions, the client’s ability to work, engage in community activities, participate in enjoyed leisure and recreation activities can be affected. Contextual factors, for example, an accessible house without stairs, a supportive spouse to perhaps drive the client to medical and rehabilitation appointments and the client’s interest in engaging in the rehabilitation process to return to work will all influence functioning, barrier reduction and ultimately, successful reintegration.

VAC determines if community integration is a barrier through a number of questions on the RROD. Statistical data from the RROD notes 67 percent of Veterans identify community-related barriers when applying for the Rehabilitation Program. It should be noted that the severity of the community barriers is measured primarily based upon the individual’s frequency of participation in hobbies, leisure activities and community activities. According to the 2008-2009 Re-establishment Survey, upon entry, 22 percent of responders entering the program report that they are “very involved” or “somewhat involved” in community activities.

Upon admission to the Rehabilitation Program, of those who indicated that they have a barrier to community integration the majority of them, 82 percent (according to RROD), classified the barrier as moderate or severe.

Effectiveness

The file review showed that of the 350 participant files reviewed, 139 identified a moderate or severe community integration barrier at the time of initial assessment. As can be seen in the Figure 3 below, those who identified community integration as a barrier, only 16 percent showed improvement, 26 percent showed no change and for 58 percent of the sample, insufficient information existed at the second assessment.

Figure 3: Change in Function for Individuals with an Identified Moderate or Severe Community Integration Barrier

Change in function for Individuals with an Identified Moderate or Severe Community Integration Barrier
Change in Community Barrier
Not Known Improved No Change
58% 16% 26%

Focus group participants were asked for their perception on whether the NVC programs helped them participate and integrate into the community. According to the results, participants tended to be divided about the extent to which VAC programs contribute to the participation or integration of Veterans in their communities. Some felt that this was definitely the case, some were unsure, and some did not think so or felt that the impact was limited.

LASS: STCL concluded that only 39 percent of NVC participants felt they had a strong or somewhat strong sense of community belonging compared to 56 percent of DP recipients, and 62 percent of Veterans not in receipt of VAC benefits.

Efficiency

An individual’s sense of community integration is directly impacted by his/her physical and psychosocial well-being, income and employment status. As noted in the File Review, measuring change in community participation barriers was difficult as there was limited information gathered.

Conclusion

  • For 67 percent of Veterans, community integration was identified as a barrier upon admission to the Rehabilitation Program.
    • Upon admission, of those who indicated that they have a barrier to community integration, the majority of them, 82 percent (according to RROD), classified the barrier as moderate or severe.
  • LASS: STCL concluded that only 39 percent of NVC Veterans felt they had sense of community belonging.
  • Community integration is directly impacted by a variety of factors, which can include: physical and psychosocial well-being, income and employment status.
  • A Veteran’s achievement of community integration may not be fully realized until some improvement occurs with other barriers.

4.4 Employment and Income

Employment

Outcome

CF members, Veterans and their families actively participate in the civilian workforce (unless totally disabled or retired) as a result of access to employment-related supports in the form of vocational assistance, training and job placement assistance.

Income

Outcome

CF members, Veterans and their families have a level of income adequate to meet basic needs as a result of enhanced employment opportunities provided by job placement assistance, and access to employment enhancing supports such as re-training opportunities as part of vocational rehabilitation.

Overview

The outcomes for employment and income are directly related. The employment outcome relates to individuals who participate in the employment support programs. The income provided by the employment gained is spoken to in the second outcome.

Additionally, employment and income are key determinants of health and influence individual, family and community health status. As a result, the assessment of these two outcomes has been combined and is reported in the section below.

The transition to civilian life is very important to CF members who are releasing at an age or time that may require many years of productive employment in the civilian sector. The objective of VAC’s employment supports is to provide vocational assistance, training, and career transition services to individuals so they have the necessary knowledge, skills, and a plan to prepare them for obtaining suitable civilian employment.

Employment supports, specifically the CTS Program (formerly called the Job Placement Program) and the Rehabilitation Program’s Vocational component were established as part of the NVC to assist CF members/Veterans with the transition from a military career to the civilian workforce. At the time of the evaluation, the CTS was designed for non-medically releasing members or Veterans requiring assistance to make the transition to the civilian labour force, while the vocational rehabilitation services supports both medically released Veterans and Veterans with a service-related barrier to re-establishment.3

VAC has not formally defined what is considered “income adequate to meet basic needs.” However, a goal of the CTS Program is that Veterans obtain suitable employment which is realistic in light of each individual’s aspirations, qualifications, experience, education and intended geographic area of residence. CTS tracks and compares the Veteran’s military salary and civilian job salary. The income goal for vocational rehabilitation services is to assist individuals in obtaining “suitable employment.” As outlined in VAC policy, the definition for “suitable gainful employment” includes pay equal to at least 66.67 percent of the Veteran’s pre-release income. According to the 2008-2009 Re-establishment Survey, upon entry, 90 percent of individuals report they are able to meet or exceed their basic living expenses.

Both the CTS and vocational rehabilitation services are delivered through national contractors. Since the NVC was implemented in 2006, various interim vocational rehabilitation specialists have been providing services, until a single national provider could be put in place to ensure more consistent service delivery. The use of multiple interim providers was challenging for evaluation purposes due to the lack of complete data. In April 2009, a national contractor began to deliver vocational services for VAC; however, participants had the option of continuing with their interim service provider or switching to the national contractor. The CTS contract started in October 2007.

The CTS Program is divided into three components: Job Search and Transition Workshops, Individual Career Counselling, and Job Finding Assistance. Success is measured by participants’ obtaining the knowledge and skills necessary to gain civilian employment as well as obtaining suitable employment.

The workshop component is usually delivered on or near CF bases, to small groups of 12-15 members interested in releasing (2,217 CF members or Veterans attended workshops within the evaluation period). The CTS assists with identifying transferable skills, résumé creation, interview training, advice on managing a civilian career, and information on self-employment. Individual career counselling and job finding assistance are the two main components of the program. The career counselling component provides more in-depth assistance including aptitude and interest testing, job market research and analysis in the intended community of residence, qualifications for various fields, finalization of a résumé, and a detailed career transition plan. The job finding component assists participants in seeking suitable civilian employment through understanding of local markets, networking opportunities, identifying interview opportunities with recruiters and a job bank of hiring companies.

Vocational rehabilitation is designed to identify and achieve an appropriate occupational goal for a person with a physical or a mental health problem, given their state of health and the extent of their education, skills and experience. Vocational rehabilitation services include but are not limited to: vocational evaluations and counselling; education and training; child care; work place ergonomic assessment and modification; and job finding/placement services.

CF members, with a wide range of skill sets, experiences, and years of service, release for a number of reasons. While the average age of release is 36 years, NVC employment support programs assist those as young as 18 years of age to those over 60. Three-quarters of releasing members are married/common-law. Approximately half of all releases have education that is at the high-school level or less. Participants’ employment interests span from looking for part-time work, to remaining active during retirement, to requiring a new career to support a young family. The CF releases 4,000-6,000 members a year in medical, voluntary and other categories. Table 5 depicts release data for 2008-2009 and 2009-2010.

Table 5 - Release Data 2008-2010
Year Total Releases Medical Voluntary and Other
2008/09 6,195 1,058 5,137
2009/10 5,237 915 4,322

Data the evaluation team reviewed indicated that over the last four years an average of 33 percent of releasing CF members who attended a transition interview indicated that they were seeking work upon finishing employment with the Canadian Forces. Also, the File Review reported that for those participating in the Rehabilitation Program, 92 percent presented with a barrier in vocational roles (some have more than one barrier) meaning they were unable to work, had difficulty on the job or keeping a job or were not work ready.

The LASS: STCL reported that at the time of the survey, 45 percent of NVC participants were working compared to 75 percent of Veterans who are not in receipt of VAC benefits. Considering the percentage of individuals who worked at any point since release, 76 percent of NVC participants and 93 percent of Veterans who are not in receipt of VAC benefits have worked at some point since release. This reinforces the need for employment support to aid individuals in obtaining employment. VAC’s vocational rehabilitation model was designed to identify any barriers and limitations which are preventing Veterans with disabilities from obtaining suitable gainful employment opportunities and to find solutions to increase their employability, based on their previous work experience, education, skills and functional capacity in relation to their disability.

The LASS Income Study provided the evaluation with summary income information for the 36,638 CF members who released between 1998 and 2007 (excludes VAC disability benefits and includes a small amount of VAC Earnings Loss benefits). From this group the average post release income for CF members was $55,800; however, there are substantial income differences among Veteran groups, as illustrated in Graph1 below. In particular, Veterans who became rehabilitation participants earned an average of $38,400 post release, which is only 58 percent of their pre-release salary and below VAC’s desired outcome.

Change in Income - LASS Study

Change in Income - LASS Study
Change in Income - LASS Study
All Released Veterans NVC Participants Rehabilitation Participants Veterans not in receipt of VAC benefits
blue Pre-Release $62,300 $71,500 $65,700 $57,900
red Post-Release $55,800 $48,700 $38,400 $55,400
yellow% of Pre-Release 90% 68% 58% 96%

Note: Income expressed in 2007 constant dollars

This information highlights the financial challenges that injured CF members are experiencing post release and supports comments from field staff that often the most immediate concern for Veterans who present to NVC is current financial obligations. LASS:STCL found that approximately 57 percent of NVC participants were satisfied with their financial situation compared to 76 percent of released Veterans not in receipt of benefits. Overall, the data illustrates the important role of employment supports for participants in the Rehabilitation Program.

Effectiveness

Of the 880 individuals who were accepted into the career counselling or job finding components of CTS from April 1, 2007 to March 31, 2010, 9 percent (77) have officially completed and obtained employment through the program. Income data was available for 22 of the 77 and showed that 15 were earning a salary equal to or higher than the Veteran’s military salary.

The low numbers who have completed may be partially explained by the significant numbers of participants (250) who are in suspended status. The majority of these individuals have been suspended at their own request. The 2008-2009 Re-establishment Survey noted that 62 percent of individuals entering the program are currently employed. It would appear that individuals are self-suspending because they are already employed but want the option of returning to the program if their current job is unsatisfactory. Generally, individuals are requesting the suspension as there is a two-year time limit to apply and if they cancelled or did not apply within their eligibility period, they cannot come back into the program without a new period of CF service.

Participants who attended focus group sessions expressed satisfaction with the CTS, with many saying they were very satisfied with the program, specifically with résumé writing/preparation and preparing for a job interview. There was also consensus that the CTS helped identify and utilize existing skills, including specialized military career skills that could be transferred to a new civilian career. Additionally, feedback collected by the national contractor of CTS also indicates participants are very satisfied with the program.

Of the 4,197 individuals who accessed the Rehabilitation Program, 1,296 (31 percent) of them accessed vocational rehabilitation and assistance services through an interim contractor, the national contractor, or both. The number of individuals accessing the program steadily increased each year, with a significant increase in the year the national contractor began operations. Approximately 13 percent of vocational rehabilitation participants completed the program as of March 31, 2010, with the remainder still receiving services. The nine individuals who completed through the national contractor obtained employment. Employment-related data from the interim providers is not captured and, therefore, not available to evaluate. Also, income data was not available for individuals who found employment to indicate if the goal of gainful employment was being met. Going forward, regular reports identifying income are to be generated by the national contractor.

Analysis of those Veterans participating in the employment support programs indicates that program uptake is higher for those releasing in their 40s and 50s than individuals in their 20s. Thirty-three percent (33 percent) of members being released by DND have five years or less service and are in their 20s; however, these individuals are not utilizing the programs to the same extent as older releasing members. The graph below depicts an age comparison of releasing CF members and the age of both CTS and Vocational Rehabilitation participants.

Graph 2: Age of Participants versus Age at Release for CTS and Vocational Rehabilitation

Age of Participants versus Age at Release for CTS and Vocational Rehabilitation
Age of Participants vs. Age at Release
20-29 30-39 40-49 50-59 60+
CTS Participation 12% 17% 43% 24% 3.25%
Age at release 30% 23% 35% 13%
Vocational Rehabilitation 8% 22% 45% 22% 3%

Additionally, vocational assistance is provided to eligible spouses/common-law partners or survivors in order to restore their earning capacity to a reasonable level. Fifty-nine spouses and/or survivors had accessed vocational assistance as of March 31, 2010. The NVC Evaluation Phase I reviewed services for families and noted that the CTS was not available to spouses. It is available to survivors; however, only two have participated.

With Veterans and their families relocating multiple times throughout their careers and upon release, spouses could benefit from CTS.

When issues were clear cut (such as barrier in vocational role) and established resources were available, individuals appeared to be serviced expediently and good outcomes were achieved. For those individuals in the sample who completed the program, one of the main reasons was because they showed improvement in vocational barriers.

Two hundred and twenty-two (222) individuals have completed the career counselling component and 28 have completed the job finding assistance component. Nearly 40 percent (340) of all eligible individuals had a suspended status since they were first deemed eligible for the program, and more than 45 percent of individuals currently in the program are in suspended status. Tables in Appendices L and M detail these activities within the CTS and vocational rehabilitation services. This unintended impact may be a result of Veterans having only a two-year eligibility window from their date of release in which to apply to the program and Veterans want to maintain their eligibility in case at some point in the future they need the services of the program.

Since the introduction of the NVC, there have been 162 (13 percent) participants who have completed vocational rehabilitation; however, almost all of these participants completed under interim contracts where income information was not collected. This means VAC has no information regarding whether participants are meeting the outcome.

Efficiency

While the NVC Evaluation Phase I reported that “some instances of overlap were identified with the Rehabilitation and Job Placement programs” in reference to similarities with other federal departments' programs, current findings indicate there are also opportunities to reassess the delivery of the CTS and vocational rehabilitation services within VAC.

The eligibility requirements are different between the CTS and vocational rehabilitation services; however, there are similarities in the objectives and outcomes (i.e. civilian employment and adequate income) and also in the services offered. The delivery method for each program is via a national contractor.

The workshops offered by the CTS Program are an efficient means of delivering services to CF members. The cost per individual is approximately (*4) , and they fulfill the needs of most releasing CF members. The average cost per individual for the career counselling and job finding components is approximately (*4). This is significantly higher than the workshop component and is not efficient for the number of individuals participating in and completing the program.

As of March 31, 2010, approximately (*4) has been spent on vocational rehabilitation services [not including salary and Operations and Maintenance (O&M) costs which could not be broken out from overall Rehabilitation Program costs]. The average cost per individual to date for vocational services is approximately (*4); however, over 80 percent of individuals are still in the program and will continue to incur costs.

Conclusion

  • 9 percent (77) of individuals in CTS have completed and obtained employment.
    • 2,217 have attended CTS Workshops.
  • 13 percent (162) completion rate in vocational rehabilitation.
  • The workshops demonstrate value for money, and appear to fill the need for non-medical releasing CF members who are seeking employment. The other two components of the program are not well attended (compared to forecasts), and do not demonstrate value for money, especially in comparison to the workshops.
  • The CTS needs assessment identified as part of the Management Action Plan for NVC Phase I evaluation is yet to be finalized.
  • There is a need to closely monitor utilization and expenditures pertaining to the contract which came into effect on October 1, 2010.
  • The file review noted that vocational rehabilitation participants appear to be well served when issues are clear cut and established resources are available.
  • Rehabilitation Program participants experience a noticeable decrease in income compared to their pre-release income and were identified as being at greatest risk of low income.
  • Preliminary results of participants' salary upon completion of the CTS Program are inconclusive.

Recommendation 4

It is recommended that the ADM, Service Delivery, in advance of retendering associated with the CTS Program, confirm the needs of the target population and, if necessary, analyze options for program design and delivery. (Essential)

Management Response

Management agrees with this recommendation and the requirement for further program monitoring and analysis given that:

  • a new contract, where payments are based on the cost of services rendered, came into effect on October 1, 2010 and therefore provides a significantly different cost structure;
  • significant changes were made to the CTS Program in March 2010 to facilitate access to individual career counselling and job finding assistance to serving CF members prior to their release from the CF; the impact of these changes must be measured; and
  • additional factors, such as future release volumes, the pending withdrawal from the Afghanistan mission, future labour market conditions, the country’s future economic situation and the government’s desire to offer services targeted specifically to CF members and Veterans, must all be factored into the decision-making regarding CTS programming.

With regard to instances of overlap between CTS and services delivered by other federal organizations, VAC had numerous consultations with Human Resources and Social Development Canada (HRSDC) and Department of National Defence (DND) during and after New Veterans Charter program design. It was determined that VAC was the appropriate department to deliver career transition services to CF members and Veterans. DND does not have the mandate to serve Veterans, and HRSDC devolved all career-related services to the provinces and territories. Through Labour Market Development Agreements, Canada provides funding from the Employment Insurance Account to provinces /territories for their employment programs and services for Employment Insurance-eligible individuals and other unemployed Canadians. Consequently there is no federal department, other than VAC, that provides career transition services to CF members. The provincial/territorial programs are mandated for the unemployed. CF members have clearly indicated that their second career preparations should start before they release from the military, and therefore while they are still employed. This is congruent with the objective of helping CF members make a smooth transition from military to civilian life, and providing services that are specifically targeted to them as CF members.

Significant changes have been made to the CTS program. Prior to early 2010, CF members could attend CTS Workshops at any point in their career but could only attend the Individual Career Counselling and Job Finding components of the program once they had submitted a request for release. Effective March 2010, VAC removed this requirement, thereby allowing CF members to access CTS early enough in the career planning process. This allows sufficient time for participants to obtain identified qualifications, education, and certifications required to support their civilian career choice, before they release from the military. CF members who participate in the Workshop component can now move directly to the Individual Career Counselling and Job Finding components of the program, which was not the case previously. This may significantly affect the participation rate in the Individual Career Counselling and Job Finding components of the program.

Considering all of these factors, Management feels that a full review of the CTS Program should be conducted, beginning one year after implementation of the current contract. The review would provide data based on the current program design and contract structure that would form the basis for decision making with respect to the options for the design and delivery of the CTS Program.

Management Action Plan
Corrective Action(s) to be taken OPI (Office of Primary Interest) Target Date
Undertake a comprehensive review, prior to contract expiry, to inform decision-making with respect to options for CTS Program design and delivery. Service Delivery Management September 30, 2011
Present report at SMC for a decision on the future of CTS program design and delivery. Service Delivery Management December 31, 2011

1 OSI is defined as "any persistent psychological difficulty resulting from operational duties performed while serving with the Canadian Forces"

2SDAs are specific geographic areas outside Canada where members are exposed to conditions of "elevated risk." An "elevated risk" is a level of risk higher than that normally associated with service in peacetime.

3CTS was designed for non-medically releasing CF members within two years from release from service.

  • Since Phase I, the program area has undertaken activities to increase participation:
    1. 9,000 letters and application forms were sent to Veterans who were potentially eligible, informing them of the program and encouraging them to apply. The response from the mail-out resulted in two hundred additional applications.
  • Changes to eligibility requirements have also been made:
    1. In March 2010 still-serving members are now eligible for all components of CTS.
    2. CF members no longer have to sign an Intent to Release form
  • In October 2010, a new contract and pricing structure for CTS came into effect.

4"Protected from disclosure in accordance with the provisions of the Access to Information Act."

Date modified: