User login
Suicide is a national public health concern that affects thousands of US individuals and families, with repercussions that reverberate through entire communities. In 2019, there were 47,500 US deaths by suicide, which accounted for about 1 death every 11 minutes.1 Suicide remains the tenth leading cause of death in the United States and has been part of the top 12 leading causes of death since 1975.2 Unfortunately, this trend has worsened; suicide rates have increased by 35% from 1999 to 2018.3 One particularly vulnerable population is US veterans who accounted for 13.8% of all suicide deaths in 2018.4 Among veterans, the suicide death average increased from 16.6 per day in 2005 to 17.6 in 2018.4 Furthermore, veterans experiencing homelessness are 5 times more likely to attempt suicide and 2.5 times more likely to have suicidal ideation compared with veterans without a history of homelessness.4 Suicide is a significant issue among veterans experiencing homelessness: Veterans account for about 11% of the overall US homeless population.5
Recent data suggest opportunities for suicide risk assessment in the primary care setting. A study from the Veterans Health Administration (VHA) Office for Suicide Prevention found that in 2014 an average of 20 veterans died by suicide every day and 6 of the 20 (30%) on average used VHA services within the prior year.6 Similarly, a review of 40 studies on suicide found that 45% of suicide victims had contact with their primary care practitioner (PCP) within 1 month of suicide, and 75% of victims had contact within the year of suicide.7 An analysis of depression screening in 2008/2009 using Patient Health Questionnaire-2 (PHQ-2) or Patient Health Questionnaire-9 (PHQ-9) at 3 large western US Department of Veterans Affairs (VA) medical centers found that 55% were screened for depression.8 The VA has made suicide prevention a top priority and supports the established US goal of reducing annual suicide deaths by 20% by 2025.9 Given key opportunities for suicide risk assessment in the primary care setting, the VHA Office of Mental Health and Suicide Prevention implemented a national, standardized process for suicide risk assessment on October 1, 2018.10,11
The VA approach to suicide screening, evaluation, and documentation has evolved over time. Between October 2018 and December 2020, the process was augmented to include 3 stages embedded into the electronic health record (EHR): a primary screen (PHQ-2 with Item 9 from the PHQ-9 [PHQ-2+I9]), a secondary screen (Columbia-Suicide Severity Rating Scale [C-SSRS]), and a tertiary screen (Comprehensive Suicide Risk Evaluation [CSRE]). The primary screen consisted of the depression screening using the PHQ-2 with the addition of I9 asking about suicidal ideation. The secondary screening, or C-SSRS, included 8 questions to elaborate on suicidal ideation, intent, plan, and any history of suicidal attempts or preparatory behaviors. The tertiary screen consisted of the CSRE, a questionnaire developed internally by the VA in 2018 to further evaluate the veteran’s suicidal thoughts, attempts, warning signs, risk factors, protective factors, and reasons for living. The goal of the screenings was to identify veterans at risk of suicide, assess risk severity, and to individually tailor risk mitigation strategies for safe disposition. These risk categories were developed by the regional Mental Illness Research, Education and Clinical Center, which suggested treatment strategies, such as hospitalization or close outpatient follow-up.12,13
The Homeless Patient Aligned Care Team (HPACT) clinic at the West Los Angeles VA Medical Center (WLAVAMC) in California, one of the largest VA homeless clinics in the country and 1 of 7 national VA Office of Academic Affiliation Centers of Excellence in Primary Care Education training programs implemented the standardized tools for suicide risk screening and quality improvement (QI). The HPACT clinic is an interprofessional team, including primary care, mental health, social work, pharmacy, and peer support, that is adjacent to the WLAVAMC general primary care clinics. The team collaboratively addresses both medical and psychosocial needs of veterans with a focus on the Housing First Model, an approach that prioritizes ending homelessness while addressing all factors associated with veterans' health and well-being. After 1 year of stable housing, veterans graduate to the WLAVAMC general primary care clinics.
Given the vulnerability of veterans experiencing homelessness, the clinic leadership identified suicide risk screening as a high priority initiative and created a taskforce to oversee effective implementation of clinic screening efforts. An interprofessional team of nurse practitioners (NPs), pharmacists, physicians, psychologists, social workers (SWs), and trainees formed to improve screening efforts and use the QI principles to guide analysis and intervention. The team wrote the following SMART (Specific, Measurable, Achievable, Relevant, and Time-bound) Aim statements: (1) ensure > 90% of eligible patients receive a primary screen; (2) ensure > 90% of positive primary I9 screens receive subsequent screenings within 24 hours; and (3) increase staff comfort and familiarity using the screening tools. This article examines the results of the screening initiative 1-year postimplementation, describes difficulties faced, and suggests strategies that might be used to overcome those challenges.
Methods
This QI analysis was exempt from institutional review board review. Prior to the standardized national suicide risk assessment rollout of October 1, 2018, the QI team met to review and understand the workflow to be implemented into the HPACT clinic. To describe the initial screening process, the new suicide risk assessment consisted of primary, secondary, and tertiary screens that would warrant subsequent intervention by clinicians if positive (Figure 1). The primary screen included the PHQ-2+I9 questionnaire (PHQ-2 for depression and I9 for suicidal ideation). If either were positive, follow-up questionnaires were required. Of note, patients with a prior depression diagnosis, cognitive impairment defined at a severity of moderate or greater based on clinician evaluation and judgement, or life expectancy < 6 months were exempt from screening because, by definition, they had theoretically already been screened and classified as under surveillance.
A positive I9 response prompted a secondary screen using C-SSRS. A positive secondary screen prompted a tertiary screen using CSRE. If the PHQ-2 screening was positive but I9 was negative, the standard follow-up depression clinical reminder was used. Any clinical staff member could perform the primary screen, including licensed vocational nurses (LVNs), registered nurses (RNs), and SWs in any setting (eg, emergency department, primary care, inpatient services). The secondary and tertiary screens required completion by a licensed clinician. RNs were able to perform the secondary screen but not the tertiary screen.
The HPACT clinic serves approximately 3000 patients by 50 staff and trainees divided into 2 teams. LVNs and RNs were tasked to conduct the primary screen as part of their initial clinic check-in. If the primary screen was positive for scheduled patients, LVNs notified a PCP to complete the secondary screen. For unscheduled patients, RNs conducted a primary screen and, if positive, a secondary screen. If the secondary screen was positive, a tertiary screen was performed by mental health practitioners or SWs, or PCPs if the former were unavailable. SWs, mental health practitioners, and PCPs were colocated in the clinic, which allowed for safe and convenient warm handoffs between clinicians.
During this process, the interprofessional team overseeing the suicide screening implementation efforts in the HPACT clinic met in-person biweekly beginning 1 month prior to the October 1, 2018 implementation. QI tools, including flowcharts and root cause analyses, were used to analyze feedback on efficient workflow and optimize staff responsibilities. A survey assessed staff comfort and familiarity using the suicide screening tools. Informal interviews were conducted with a representative from each stage of patient care to facilitate interprofessional participation and to troubleshoot any issues. Process flowcharts that clearly delineated staff roles based on current clinic workflow and the recommendations set forth by the new process were distributed at an initial staff meeting. The process flowchart was updated after staff feedback and distributed again along with a review of the C-SSRS and CSRE at an all-staff meeting in February 2019. The QI team continued to meet to formally evaluate their SMART Aims and to identify factors driving the success and failure of the implementation.
The VA Informatics and Computing Infrastructure (VINCI) provided project data after a formal request was submitted for this analysis. At the direction of the local QI team, the VINCI team provided aggregate patient counts derived from individual patient data in the VA Corporate Data Warehouse. The data analyzed are frequencies and proportions; no bivariate or multivariate statistics were performed.
Results
During the project year, the HPACT clinic had 2932 unique patients assigned to primary care. Of those veterans, 533 (18%) were exempt from screening by protocol. Of the remainder, staff screened 1876 (64%) of eligible veterans for suicide risk (Figure 2), which did not meet the SMART Aim of screening > 90% of eligible veterans. For the follow-up screens, using a QI dashboard designed for reviewing I9 and C-SSRS results, the QI team reviewed a convenience sample of 5 provider panels and identified 34 positive I9 screens. Twenty of those 34 patients (59%) received a C-SSRS within 24 hours of the positive I9, which did not meet the SMART Aim of ensuring > 90% of primary I9 screens had subsequent C-SSRS screening within 24 hours.
Of the veterans screened, 1,271 (43%) had their screening performed outside of the HPACT primary care team assigned, while 605 (21%) patients had their screening performed by an HPACT member. Most of the screening that occurred outside of the assigned primary care team occurred in other physical settings, including other VA facilities.
Of the 523 (18%) patients who were not screened, 331 (11%) patients had no visit to the HPACT clinic and 132 (5%) empaneled patients did not visit any VA site within the 1-year period. There were 192 (7%) patients who were not screened that had a visit to HPACT while 19 (1%) of those patients declined screening. A total of 184 (6%) patients were not screened and thus were considered true missed opportunities. This group of patients were eligible for screening but did not undergo screening in the HPACT clinic or any other VA setting despite visiting the VA.
The QI team created a fishbone diagram to identify opportunities to improve screening rates and patient care (Figure 3). Using the fishbone tool, the QI team identified 5 main categories limiting complete uptake of suicide risk assessment at the HPACT clinic: health record factors, communication, clinician buy-in, system factors, and patient factors. Among the most salient barriers to use of the screening tool, the EHR system needed to be refreshed after a positive screen to be reminded of the next step, requiring close communication during patient handoffs. Handoff was confusing as there was no dedicated process to communicate positive screen information. Clinicians were concerned that completing the process, especially the tertiary screen, would be time consuming and burdensome in an already busy clinic; some clinicians were uncomfortable discussing the topic of suicide as they did not feel they had the expertise to address a positive screen. In addition, some patients were reluctant to answer the screen honestly due to past hospitalizations or concerns about stigma.
Discussion
Though the QI project failed to meet the SMART Aim of ensuring > 90% of eligible patients received a primary screen for suicide risk and > 90% of positive primary I9 screens received subsequent screenings within 24 hours, the results highlight effective practices and barriers for implementation of wide-scale EHR-based interventions for suicide assessment. Most missed screening opportunities were due to patients being lost to follow-up over the duration of the project, which is a challenge faced in this patient population. A recent analysis of the national rollout of this screening program found that 95% of eligible veterans with a visit to the VA in the first year of the program received screening.14 In a post hoc analysis using the same eligibility criteria, the rate of screening for this project was 83%. Reflecting on the data from this national cohort compared with the HPACT clinic, this brings to light potential circumstances that may be unique to veterans experiencing homelessness compared with the general veteran population, for instance, the level of engagement may be lower among veterans experiencing homelessness, though this is beyond the scope of this article. Nonetheless, promoting interprofessional collaboration, visualizing effective process flows, establishing clear lines of communication and roles for involved staff, and opening avenues for continuous feedback and troubleshooting are all potentially effective interventions to improve suicide screening rates within the veteran population.
This HPACT clinic initiative aimed to determine how a new screening process would be implemented while identifying potential areas for improvement. Surprisingly, 43% of patients who were screened had their screening performed outside of the HPACT clinic, most often in the inpatient setting at other WLAVAMC clinics or other VA systems. It is possible that due to the nature of the patient population that the HPACT clinic serves with intensive service needs, these patients have wider geographic and clinical location use than most clinic populations due to the transient nature of patients with housing insecurity. What is encouraging, however, is that through this systemwide initiative, there is an impetus to screen veterans, regardless of who performs the screening. This is particularly meaningful given that rates of depression screening may be as low as 4% among PCPs.15 During implementation, the QI team learned that nearly 18% of the empaneled HPACT patients were exempt from screening. The exempt patients do not have an active clinical reminder for depression screens. Instead, these patients are receiving mental health surveillance and specialty treatment, during which continuous monitoring and assessment for suicidal ideation and risk of suicide are performed. Additionally, an EHR-based factor that also may limit appropriate follow-up and contribute to missed opportunities is that secondary and tertiary screens do not populate until the EHR was refreshed after positive primary screens, which introduces human error in a process that could be automated. Both RNs and PCPs may occasionally miss secondary and tertiary screens due to this issue, which continues to be a barrier. Given the high risk HPACT clinic population, the QI team encouraged staff members to frequently screen patients for suicidal ideation regardless of clinical reminders. A consideration for the future would be to identify optimal frequency for screening and to continue to validate assessment methods.
Finally, while the percentage of patients who were considered missed opportunities (visited the HPACT clinic but were not screened) was relatively small at 6% of the total panel of patients, this number theoretically should be zero. Though this project was not designed to identify the specific causes for missed opportunities, future QI efforts may consider evaluating for other potential reasons. These may include differing process flows for various encounters (same-day care visits, scheduled primary care visit, RN-only visit), screening not activating at time of visit, time constraints, or other unseen reasons. Another important population is the 11% of patients who were otherwise eligible for screening but did not visit the HPACT clinic, and in some cases, no other VA location. There are a few explanatory reasons centered on the mobility of this population between health systems. However, this patient population also may be among the most vulnerable and at risk: 62% of veteran suicides in 2017 had not had a VA encounter that year.13 While there is no requirement that the veteran visit the HPACT clinic annually, future efforts may focus on increasing engagement through other means of outreach, including site visits and community care involvement, knowing the nature of the sporadic follow-up patterns in this patient population. Future work may also involve examining suicide rates by primary care clinic and triage patterns between interprofessional staff.
Limitations
Due to the limited sample size, findings cannot be generalized to all VA sites. The QI team used retrospective, administrative data. Additionally, since this is a primary care clinic focused on a specialized population, this result may not be generalizable to all primary care settings, other primary care populations, or even other homeless primary care clinics, though it may establish a benchmark when other clinics internally examine their data and processes.
Conclusions
Improving screening protocols can lead to identification of at-risk individuals who would not have otherwise been identified.16,17 As the US continues to grapple with mental health and suicide, efforts toward addressing this important issue among veterans remains a top priority.
Acknowledgments
Thank you to the VAGLAHS Center of Excellence in Primary Care Education faculty and trainees, the HPACT staff, and the VA Informatics and Computing Infrastructure (VINCI) for data support.
1. Centers for Disease Control and Prevention. Facts about suicide. Reviewed August 30, 2021. Accessed December 13, 2021. https://www.cdc.gov/suicide/facts/index.html
2. Centers for Disease Control and Prevention. Preventing suicide: a technical package of policies, programs, and practices. Published 2017. Accessed December 13, 2021. https://www.cdc.gov/violenceprevention/pdf/suicideTechnicalPackage.pdf
3. Centers for Disease Control and Prevention. Increase in suicide mortality in the United States, 1999-2018. April 8, 2020. Accessed December 13, 2021. https://www.cdc.gov/nchs/products/databriefs/db362.htm
4. US Department of Veterans Affairs, Office of Mental Health and Suicide Prevention. 2020 National Veteran Suicide Prevention Annual Report. Published November 2020. Accessed December 13, 2021. https://www.mentalhealth.va.gov/docs/data-sheets/2020/2020-National-Veteran-Suicide-Prevention-Annual-Report-11-2020-508.pdf
5. Culhane D, Szymkowiak D, Schinka, JA. Suicidality and the onset of homelessness: evidence for a temporal association from VHA treatment records. Psychiatr Serv. 2019;70(11):1049-1052. doi:10.1176/appi.ps.201800415
6. US Department of Housing and Urban Development. The 2015 annual homeless assessment report (AHAR) to Congress. Published November 2015. Accessed December 13, 2021. https://www.hudexchange.info/resources/documents/2015-AHAR-Part-1.pdf
7. US Department of Veterans Affairs, Office of Suicide Prevention. Suicide among veterans and other Americans 2001-2014. Published August 3, 2016. Updated August 2017. Accessed December 13, 2021. https://www.mentalhealth.va.gov/docs/2016suicidedatareport.pdf
8. Dobscha SK, Corson K, Helmer DA, et al. Brief assessment for suicidal ideation in OEF/OIF veterans with positive depression screens. Gen Hosp Psychiatry. 2013;35(3):272-278. doi:10.1016/j.genhosppsych.2012.12.001
9. Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care providers before suicide: a review of the evidence. Am J Psychiatry. 2002;159(6):909-916. doi:10.1176/appi.ajp.159.6.909
10. US Department of Veterans Affairs. National strategy for preventing veteran suicide 2018-2028. Accessed December 13, 2021. https://sprc.org/sites/default/files/resource-program/VA_National-Strategy-for-Preventing-Veterans-Suicide2018.pdf
11. US Department of Veterans Affairs. VA suicide prevention efforts. Published July 2019. Accessed December 15, 2021. https://www.mentalhealth.va.gov/suicide_prevention/docs/VA_Suicide_Prevention_Program_Fact_Sheet_508.pdf
12. Wortzel H, Matarazzo B, Homaifer B. A model for therapeutic risk management of the suicidal patient. J Psychiatr Pract. 2013;19(4):323-326. doi:10.1097/01.pra.0000432603.99211.e8
13. US Department of Veterans Affairs. VA/DoD clinical practice guidelines for the assessment and management of patients at risk for suicide. Provider summary version 2.0. Published 2019. Accessed on December 3, 2020. https://www.healthquality.va.gov/guidelines/MH/srb/VADoDSuicideRiskFullCPGFinal5088919.pdf
14. Bahraini N, Brenner LA, Barry C, et al. Assessment of rates of suicide risk screening and prevalence of positive screening results among US veterans after implementation of the Veterans Affairs suicide risk identification strategy. JAMA Netw Open. 2020;3(10):e2022531. doi:10.1001/jamanetworkopen.2020.22531
15. Akincigil A, Matthews EB. National rates and patterns of depression screening in primary care: results from 2012 and 2013. Psychiatr Serv. 2017;68(7):660-666. doi:10.1176/appi.ps.201600096
16. Posner K, Brent D, Lucas C, et al. Columbia-suicide severity rating scale (C-SSRS). Columbia University Medical Center, New York, NY. 2008. Accessed December 3, 2020. https://cssrs.columbia.edu/wp-content/uploads/C-SSRS-Screening_AU5.1_eng-USori.pdf
17. Boudreaux ED, Camargo CA Jr, Arias SA, et al. Improving suicide risk screening and detection in the emergency department. Am J Prev Med. 2016;50(4):445-453. doi:10.1016/j/amepre.2015.09.029
Suicide is a national public health concern that affects thousands of US individuals and families, with repercussions that reverberate through entire communities. In 2019, there were 47,500 US deaths by suicide, which accounted for about 1 death every 11 minutes.1 Suicide remains the tenth leading cause of death in the United States and has been part of the top 12 leading causes of death since 1975.2 Unfortunately, this trend has worsened; suicide rates have increased by 35% from 1999 to 2018.3 One particularly vulnerable population is US veterans who accounted for 13.8% of all suicide deaths in 2018.4 Among veterans, the suicide death average increased from 16.6 per day in 2005 to 17.6 in 2018.4 Furthermore, veterans experiencing homelessness are 5 times more likely to attempt suicide and 2.5 times more likely to have suicidal ideation compared with veterans without a history of homelessness.4 Suicide is a significant issue among veterans experiencing homelessness: Veterans account for about 11% of the overall US homeless population.5
Recent data suggest opportunities for suicide risk assessment in the primary care setting. A study from the Veterans Health Administration (VHA) Office for Suicide Prevention found that in 2014 an average of 20 veterans died by suicide every day and 6 of the 20 (30%) on average used VHA services within the prior year.6 Similarly, a review of 40 studies on suicide found that 45% of suicide victims had contact with their primary care practitioner (PCP) within 1 month of suicide, and 75% of victims had contact within the year of suicide.7 An analysis of depression screening in 2008/2009 using Patient Health Questionnaire-2 (PHQ-2) or Patient Health Questionnaire-9 (PHQ-9) at 3 large western US Department of Veterans Affairs (VA) medical centers found that 55% were screened for depression.8 The VA has made suicide prevention a top priority and supports the established US goal of reducing annual suicide deaths by 20% by 2025.9 Given key opportunities for suicide risk assessment in the primary care setting, the VHA Office of Mental Health and Suicide Prevention implemented a national, standardized process for suicide risk assessment on October 1, 2018.10,11
The VA approach to suicide screening, evaluation, and documentation has evolved over time. Between October 2018 and December 2020, the process was augmented to include 3 stages embedded into the electronic health record (EHR): a primary screen (PHQ-2 with Item 9 from the PHQ-9 [PHQ-2+I9]), a secondary screen (Columbia-Suicide Severity Rating Scale [C-SSRS]), and a tertiary screen (Comprehensive Suicide Risk Evaluation [CSRE]). The primary screen consisted of the depression screening using the PHQ-2 with the addition of I9 asking about suicidal ideation. The secondary screening, or C-SSRS, included 8 questions to elaborate on suicidal ideation, intent, plan, and any history of suicidal attempts or preparatory behaviors. The tertiary screen consisted of the CSRE, a questionnaire developed internally by the VA in 2018 to further evaluate the veteran’s suicidal thoughts, attempts, warning signs, risk factors, protective factors, and reasons for living. The goal of the screenings was to identify veterans at risk of suicide, assess risk severity, and to individually tailor risk mitigation strategies for safe disposition. These risk categories were developed by the regional Mental Illness Research, Education and Clinical Center, which suggested treatment strategies, such as hospitalization or close outpatient follow-up.12,13
The Homeless Patient Aligned Care Team (HPACT) clinic at the West Los Angeles VA Medical Center (WLAVAMC) in California, one of the largest VA homeless clinics in the country and 1 of 7 national VA Office of Academic Affiliation Centers of Excellence in Primary Care Education training programs implemented the standardized tools for suicide risk screening and quality improvement (QI). The HPACT clinic is an interprofessional team, including primary care, mental health, social work, pharmacy, and peer support, that is adjacent to the WLAVAMC general primary care clinics. The team collaboratively addresses both medical and psychosocial needs of veterans with a focus on the Housing First Model, an approach that prioritizes ending homelessness while addressing all factors associated with veterans' health and well-being. After 1 year of stable housing, veterans graduate to the WLAVAMC general primary care clinics.
Given the vulnerability of veterans experiencing homelessness, the clinic leadership identified suicide risk screening as a high priority initiative and created a taskforce to oversee effective implementation of clinic screening efforts. An interprofessional team of nurse practitioners (NPs), pharmacists, physicians, psychologists, social workers (SWs), and trainees formed to improve screening efforts and use the QI principles to guide analysis and intervention. The team wrote the following SMART (Specific, Measurable, Achievable, Relevant, and Time-bound) Aim statements: (1) ensure > 90% of eligible patients receive a primary screen; (2) ensure > 90% of positive primary I9 screens receive subsequent screenings within 24 hours; and (3) increase staff comfort and familiarity using the screening tools. This article examines the results of the screening initiative 1-year postimplementation, describes difficulties faced, and suggests strategies that might be used to overcome those challenges.
Methods
This QI analysis was exempt from institutional review board review. Prior to the standardized national suicide risk assessment rollout of October 1, 2018, the QI team met to review and understand the workflow to be implemented into the HPACT clinic. To describe the initial screening process, the new suicide risk assessment consisted of primary, secondary, and tertiary screens that would warrant subsequent intervention by clinicians if positive (Figure 1). The primary screen included the PHQ-2+I9 questionnaire (PHQ-2 for depression and I9 for suicidal ideation). If either were positive, follow-up questionnaires were required. Of note, patients with a prior depression diagnosis, cognitive impairment defined at a severity of moderate or greater based on clinician evaluation and judgement, or life expectancy < 6 months were exempt from screening because, by definition, they had theoretically already been screened and classified as under surveillance.
A positive I9 response prompted a secondary screen using C-SSRS. A positive secondary screen prompted a tertiary screen using CSRE. If the PHQ-2 screening was positive but I9 was negative, the standard follow-up depression clinical reminder was used. Any clinical staff member could perform the primary screen, including licensed vocational nurses (LVNs), registered nurses (RNs), and SWs in any setting (eg, emergency department, primary care, inpatient services). The secondary and tertiary screens required completion by a licensed clinician. RNs were able to perform the secondary screen but not the tertiary screen.
The HPACT clinic serves approximately 3000 patients by 50 staff and trainees divided into 2 teams. LVNs and RNs were tasked to conduct the primary screen as part of their initial clinic check-in. If the primary screen was positive for scheduled patients, LVNs notified a PCP to complete the secondary screen. For unscheduled patients, RNs conducted a primary screen and, if positive, a secondary screen. If the secondary screen was positive, a tertiary screen was performed by mental health practitioners or SWs, or PCPs if the former were unavailable. SWs, mental health practitioners, and PCPs were colocated in the clinic, which allowed for safe and convenient warm handoffs between clinicians.
During this process, the interprofessional team overseeing the suicide screening implementation efforts in the HPACT clinic met in-person biweekly beginning 1 month prior to the October 1, 2018 implementation. QI tools, including flowcharts and root cause analyses, were used to analyze feedback on efficient workflow and optimize staff responsibilities. A survey assessed staff comfort and familiarity using the suicide screening tools. Informal interviews were conducted with a representative from each stage of patient care to facilitate interprofessional participation and to troubleshoot any issues. Process flowcharts that clearly delineated staff roles based on current clinic workflow and the recommendations set forth by the new process were distributed at an initial staff meeting. The process flowchart was updated after staff feedback and distributed again along with a review of the C-SSRS and CSRE at an all-staff meeting in February 2019. The QI team continued to meet to formally evaluate their SMART Aims and to identify factors driving the success and failure of the implementation.
The VA Informatics and Computing Infrastructure (VINCI) provided project data after a formal request was submitted for this analysis. At the direction of the local QI team, the VINCI team provided aggregate patient counts derived from individual patient data in the VA Corporate Data Warehouse. The data analyzed are frequencies and proportions; no bivariate or multivariate statistics were performed.
Results
During the project year, the HPACT clinic had 2932 unique patients assigned to primary care. Of those veterans, 533 (18%) were exempt from screening by protocol. Of the remainder, staff screened 1876 (64%) of eligible veterans for suicide risk (Figure 2), which did not meet the SMART Aim of screening > 90% of eligible veterans. For the follow-up screens, using a QI dashboard designed for reviewing I9 and C-SSRS results, the QI team reviewed a convenience sample of 5 provider panels and identified 34 positive I9 screens. Twenty of those 34 patients (59%) received a C-SSRS within 24 hours of the positive I9, which did not meet the SMART Aim of ensuring > 90% of primary I9 screens had subsequent C-SSRS screening within 24 hours.
Of the veterans screened, 1,271 (43%) had their screening performed outside of the HPACT primary care team assigned, while 605 (21%) patients had their screening performed by an HPACT member. Most of the screening that occurred outside of the assigned primary care team occurred in other physical settings, including other VA facilities.
Of the 523 (18%) patients who were not screened, 331 (11%) patients had no visit to the HPACT clinic and 132 (5%) empaneled patients did not visit any VA site within the 1-year period. There were 192 (7%) patients who were not screened that had a visit to HPACT while 19 (1%) of those patients declined screening. A total of 184 (6%) patients were not screened and thus were considered true missed opportunities. This group of patients were eligible for screening but did not undergo screening in the HPACT clinic or any other VA setting despite visiting the VA.
The QI team created a fishbone diagram to identify opportunities to improve screening rates and patient care (Figure 3). Using the fishbone tool, the QI team identified 5 main categories limiting complete uptake of suicide risk assessment at the HPACT clinic: health record factors, communication, clinician buy-in, system factors, and patient factors. Among the most salient barriers to use of the screening tool, the EHR system needed to be refreshed after a positive screen to be reminded of the next step, requiring close communication during patient handoffs. Handoff was confusing as there was no dedicated process to communicate positive screen information. Clinicians were concerned that completing the process, especially the tertiary screen, would be time consuming and burdensome in an already busy clinic; some clinicians were uncomfortable discussing the topic of suicide as they did not feel they had the expertise to address a positive screen. In addition, some patients were reluctant to answer the screen honestly due to past hospitalizations or concerns about stigma.
Discussion
Though the QI project failed to meet the SMART Aim of ensuring > 90% of eligible patients received a primary screen for suicide risk and > 90% of positive primary I9 screens received subsequent screenings within 24 hours, the results highlight effective practices and barriers for implementation of wide-scale EHR-based interventions for suicide assessment. Most missed screening opportunities were due to patients being lost to follow-up over the duration of the project, which is a challenge faced in this patient population. A recent analysis of the national rollout of this screening program found that 95% of eligible veterans with a visit to the VA in the first year of the program received screening.14 In a post hoc analysis using the same eligibility criteria, the rate of screening for this project was 83%. Reflecting on the data from this national cohort compared with the HPACT clinic, this brings to light potential circumstances that may be unique to veterans experiencing homelessness compared with the general veteran population, for instance, the level of engagement may be lower among veterans experiencing homelessness, though this is beyond the scope of this article. Nonetheless, promoting interprofessional collaboration, visualizing effective process flows, establishing clear lines of communication and roles for involved staff, and opening avenues for continuous feedback and troubleshooting are all potentially effective interventions to improve suicide screening rates within the veteran population.
This HPACT clinic initiative aimed to determine how a new screening process would be implemented while identifying potential areas for improvement. Surprisingly, 43% of patients who were screened had their screening performed outside of the HPACT clinic, most often in the inpatient setting at other WLAVAMC clinics or other VA systems. It is possible that due to the nature of the patient population that the HPACT clinic serves with intensive service needs, these patients have wider geographic and clinical location use than most clinic populations due to the transient nature of patients with housing insecurity. What is encouraging, however, is that through this systemwide initiative, there is an impetus to screen veterans, regardless of who performs the screening. This is particularly meaningful given that rates of depression screening may be as low as 4% among PCPs.15 During implementation, the QI team learned that nearly 18% of the empaneled HPACT patients were exempt from screening. The exempt patients do not have an active clinical reminder for depression screens. Instead, these patients are receiving mental health surveillance and specialty treatment, during which continuous monitoring and assessment for suicidal ideation and risk of suicide are performed. Additionally, an EHR-based factor that also may limit appropriate follow-up and contribute to missed opportunities is that secondary and tertiary screens do not populate until the EHR was refreshed after positive primary screens, which introduces human error in a process that could be automated. Both RNs and PCPs may occasionally miss secondary and tertiary screens due to this issue, which continues to be a barrier. Given the high risk HPACT clinic population, the QI team encouraged staff members to frequently screen patients for suicidal ideation regardless of clinical reminders. A consideration for the future would be to identify optimal frequency for screening and to continue to validate assessment methods.
Finally, while the percentage of patients who were considered missed opportunities (visited the HPACT clinic but were not screened) was relatively small at 6% of the total panel of patients, this number theoretically should be zero. Though this project was not designed to identify the specific causes for missed opportunities, future QI efforts may consider evaluating for other potential reasons. These may include differing process flows for various encounters (same-day care visits, scheduled primary care visit, RN-only visit), screening not activating at time of visit, time constraints, or other unseen reasons. Another important population is the 11% of patients who were otherwise eligible for screening but did not visit the HPACT clinic, and in some cases, no other VA location. There are a few explanatory reasons centered on the mobility of this population between health systems. However, this patient population also may be among the most vulnerable and at risk: 62% of veteran suicides in 2017 had not had a VA encounter that year.13 While there is no requirement that the veteran visit the HPACT clinic annually, future efforts may focus on increasing engagement through other means of outreach, including site visits and community care involvement, knowing the nature of the sporadic follow-up patterns in this patient population. Future work may also involve examining suicide rates by primary care clinic and triage patterns between interprofessional staff.
Limitations
Due to the limited sample size, findings cannot be generalized to all VA sites. The QI team used retrospective, administrative data. Additionally, since this is a primary care clinic focused on a specialized population, this result may not be generalizable to all primary care settings, other primary care populations, or even other homeless primary care clinics, though it may establish a benchmark when other clinics internally examine their data and processes.
Conclusions
Improving screening protocols can lead to identification of at-risk individuals who would not have otherwise been identified.16,17 As the US continues to grapple with mental health and suicide, efforts toward addressing this important issue among veterans remains a top priority.
Acknowledgments
Thank you to the VAGLAHS Center of Excellence in Primary Care Education faculty and trainees, the HPACT staff, and the VA Informatics and Computing Infrastructure (VINCI) for data support.
Suicide is a national public health concern that affects thousands of US individuals and families, with repercussions that reverberate through entire communities. In 2019, there were 47,500 US deaths by suicide, which accounted for about 1 death every 11 minutes.1 Suicide remains the tenth leading cause of death in the United States and has been part of the top 12 leading causes of death since 1975.2 Unfortunately, this trend has worsened; suicide rates have increased by 35% from 1999 to 2018.3 One particularly vulnerable population is US veterans who accounted for 13.8% of all suicide deaths in 2018.4 Among veterans, the suicide death average increased from 16.6 per day in 2005 to 17.6 in 2018.4 Furthermore, veterans experiencing homelessness are 5 times more likely to attempt suicide and 2.5 times more likely to have suicidal ideation compared with veterans without a history of homelessness.4 Suicide is a significant issue among veterans experiencing homelessness: Veterans account for about 11% of the overall US homeless population.5
Recent data suggest opportunities for suicide risk assessment in the primary care setting. A study from the Veterans Health Administration (VHA) Office for Suicide Prevention found that in 2014 an average of 20 veterans died by suicide every day and 6 of the 20 (30%) on average used VHA services within the prior year.6 Similarly, a review of 40 studies on suicide found that 45% of suicide victims had contact with their primary care practitioner (PCP) within 1 month of suicide, and 75% of victims had contact within the year of suicide.7 An analysis of depression screening in 2008/2009 using Patient Health Questionnaire-2 (PHQ-2) or Patient Health Questionnaire-9 (PHQ-9) at 3 large western US Department of Veterans Affairs (VA) medical centers found that 55% were screened for depression.8 The VA has made suicide prevention a top priority and supports the established US goal of reducing annual suicide deaths by 20% by 2025.9 Given key opportunities for suicide risk assessment in the primary care setting, the VHA Office of Mental Health and Suicide Prevention implemented a national, standardized process for suicide risk assessment on October 1, 2018.10,11
The VA approach to suicide screening, evaluation, and documentation has evolved over time. Between October 2018 and December 2020, the process was augmented to include 3 stages embedded into the electronic health record (EHR): a primary screen (PHQ-2 with Item 9 from the PHQ-9 [PHQ-2+I9]), a secondary screen (Columbia-Suicide Severity Rating Scale [C-SSRS]), and a tertiary screen (Comprehensive Suicide Risk Evaluation [CSRE]). The primary screen consisted of the depression screening using the PHQ-2 with the addition of I9 asking about suicidal ideation. The secondary screening, or C-SSRS, included 8 questions to elaborate on suicidal ideation, intent, plan, and any history of suicidal attempts or preparatory behaviors. The tertiary screen consisted of the CSRE, a questionnaire developed internally by the VA in 2018 to further evaluate the veteran’s suicidal thoughts, attempts, warning signs, risk factors, protective factors, and reasons for living. The goal of the screenings was to identify veterans at risk of suicide, assess risk severity, and to individually tailor risk mitigation strategies for safe disposition. These risk categories were developed by the regional Mental Illness Research, Education and Clinical Center, which suggested treatment strategies, such as hospitalization or close outpatient follow-up.12,13
The Homeless Patient Aligned Care Team (HPACT) clinic at the West Los Angeles VA Medical Center (WLAVAMC) in California, one of the largest VA homeless clinics in the country and 1 of 7 national VA Office of Academic Affiliation Centers of Excellence in Primary Care Education training programs implemented the standardized tools for suicide risk screening and quality improvement (QI). The HPACT clinic is an interprofessional team, including primary care, mental health, social work, pharmacy, and peer support, that is adjacent to the WLAVAMC general primary care clinics. The team collaboratively addresses both medical and psychosocial needs of veterans with a focus on the Housing First Model, an approach that prioritizes ending homelessness while addressing all factors associated with veterans' health and well-being. After 1 year of stable housing, veterans graduate to the WLAVAMC general primary care clinics.
Given the vulnerability of veterans experiencing homelessness, the clinic leadership identified suicide risk screening as a high priority initiative and created a taskforce to oversee effective implementation of clinic screening efforts. An interprofessional team of nurse practitioners (NPs), pharmacists, physicians, psychologists, social workers (SWs), and trainees formed to improve screening efforts and use the QI principles to guide analysis and intervention. The team wrote the following SMART (Specific, Measurable, Achievable, Relevant, and Time-bound) Aim statements: (1) ensure > 90% of eligible patients receive a primary screen; (2) ensure > 90% of positive primary I9 screens receive subsequent screenings within 24 hours; and (3) increase staff comfort and familiarity using the screening tools. This article examines the results of the screening initiative 1-year postimplementation, describes difficulties faced, and suggests strategies that might be used to overcome those challenges.
Methods
This QI analysis was exempt from institutional review board review. Prior to the standardized national suicide risk assessment rollout of October 1, 2018, the QI team met to review and understand the workflow to be implemented into the HPACT clinic. To describe the initial screening process, the new suicide risk assessment consisted of primary, secondary, and tertiary screens that would warrant subsequent intervention by clinicians if positive (Figure 1). The primary screen included the PHQ-2+I9 questionnaire (PHQ-2 for depression and I9 for suicidal ideation). If either were positive, follow-up questionnaires were required. Of note, patients with a prior depression diagnosis, cognitive impairment defined at a severity of moderate or greater based on clinician evaluation and judgement, or life expectancy < 6 months were exempt from screening because, by definition, they had theoretically already been screened and classified as under surveillance.
A positive I9 response prompted a secondary screen using C-SSRS. A positive secondary screen prompted a tertiary screen using CSRE. If the PHQ-2 screening was positive but I9 was negative, the standard follow-up depression clinical reminder was used. Any clinical staff member could perform the primary screen, including licensed vocational nurses (LVNs), registered nurses (RNs), and SWs in any setting (eg, emergency department, primary care, inpatient services). The secondary and tertiary screens required completion by a licensed clinician. RNs were able to perform the secondary screen but not the tertiary screen.
The HPACT clinic serves approximately 3000 patients by 50 staff and trainees divided into 2 teams. LVNs and RNs were tasked to conduct the primary screen as part of their initial clinic check-in. If the primary screen was positive for scheduled patients, LVNs notified a PCP to complete the secondary screen. For unscheduled patients, RNs conducted a primary screen and, if positive, a secondary screen. If the secondary screen was positive, a tertiary screen was performed by mental health practitioners or SWs, or PCPs if the former were unavailable. SWs, mental health practitioners, and PCPs were colocated in the clinic, which allowed for safe and convenient warm handoffs between clinicians.
During this process, the interprofessional team overseeing the suicide screening implementation efforts in the HPACT clinic met in-person biweekly beginning 1 month prior to the October 1, 2018 implementation. QI tools, including flowcharts and root cause analyses, were used to analyze feedback on efficient workflow and optimize staff responsibilities. A survey assessed staff comfort and familiarity using the suicide screening tools. Informal interviews were conducted with a representative from each stage of patient care to facilitate interprofessional participation and to troubleshoot any issues. Process flowcharts that clearly delineated staff roles based on current clinic workflow and the recommendations set forth by the new process were distributed at an initial staff meeting. The process flowchart was updated after staff feedback and distributed again along with a review of the C-SSRS and CSRE at an all-staff meeting in February 2019. The QI team continued to meet to formally evaluate their SMART Aims and to identify factors driving the success and failure of the implementation.
The VA Informatics and Computing Infrastructure (VINCI) provided project data after a formal request was submitted for this analysis. At the direction of the local QI team, the VINCI team provided aggregate patient counts derived from individual patient data in the VA Corporate Data Warehouse. The data analyzed are frequencies and proportions; no bivariate or multivariate statistics were performed.
Results
During the project year, the HPACT clinic had 2932 unique patients assigned to primary care. Of those veterans, 533 (18%) were exempt from screening by protocol. Of the remainder, staff screened 1876 (64%) of eligible veterans for suicide risk (Figure 2), which did not meet the SMART Aim of screening > 90% of eligible veterans. For the follow-up screens, using a QI dashboard designed for reviewing I9 and C-SSRS results, the QI team reviewed a convenience sample of 5 provider panels and identified 34 positive I9 screens. Twenty of those 34 patients (59%) received a C-SSRS within 24 hours of the positive I9, which did not meet the SMART Aim of ensuring > 90% of primary I9 screens had subsequent C-SSRS screening within 24 hours.
Of the veterans screened, 1,271 (43%) had their screening performed outside of the HPACT primary care team assigned, while 605 (21%) patients had their screening performed by an HPACT member. Most of the screening that occurred outside of the assigned primary care team occurred in other physical settings, including other VA facilities.
Of the 523 (18%) patients who were not screened, 331 (11%) patients had no visit to the HPACT clinic and 132 (5%) empaneled patients did not visit any VA site within the 1-year period. There were 192 (7%) patients who were not screened that had a visit to HPACT while 19 (1%) of those patients declined screening. A total of 184 (6%) patients were not screened and thus were considered true missed opportunities. This group of patients were eligible for screening but did not undergo screening in the HPACT clinic or any other VA setting despite visiting the VA.
The QI team created a fishbone diagram to identify opportunities to improve screening rates and patient care (Figure 3). Using the fishbone tool, the QI team identified 5 main categories limiting complete uptake of suicide risk assessment at the HPACT clinic: health record factors, communication, clinician buy-in, system factors, and patient factors. Among the most salient barriers to use of the screening tool, the EHR system needed to be refreshed after a positive screen to be reminded of the next step, requiring close communication during patient handoffs. Handoff was confusing as there was no dedicated process to communicate positive screen information. Clinicians were concerned that completing the process, especially the tertiary screen, would be time consuming and burdensome in an already busy clinic; some clinicians were uncomfortable discussing the topic of suicide as they did not feel they had the expertise to address a positive screen. In addition, some patients were reluctant to answer the screen honestly due to past hospitalizations or concerns about stigma.
Discussion
Though the QI project failed to meet the SMART Aim of ensuring > 90% of eligible patients received a primary screen for suicide risk and > 90% of positive primary I9 screens received subsequent screenings within 24 hours, the results highlight effective practices and barriers for implementation of wide-scale EHR-based interventions for suicide assessment. Most missed screening opportunities were due to patients being lost to follow-up over the duration of the project, which is a challenge faced in this patient population. A recent analysis of the national rollout of this screening program found that 95% of eligible veterans with a visit to the VA in the first year of the program received screening.14 In a post hoc analysis using the same eligibility criteria, the rate of screening for this project was 83%. Reflecting on the data from this national cohort compared with the HPACT clinic, this brings to light potential circumstances that may be unique to veterans experiencing homelessness compared with the general veteran population, for instance, the level of engagement may be lower among veterans experiencing homelessness, though this is beyond the scope of this article. Nonetheless, promoting interprofessional collaboration, visualizing effective process flows, establishing clear lines of communication and roles for involved staff, and opening avenues for continuous feedback and troubleshooting are all potentially effective interventions to improve suicide screening rates within the veteran population.
This HPACT clinic initiative aimed to determine how a new screening process would be implemented while identifying potential areas for improvement. Surprisingly, 43% of patients who were screened had their screening performed outside of the HPACT clinic, most often in the inpatient setting at other WLAVAMC clinics or other VA systems. It is possible that due to the nature of the patient population that the HPACT clinic serves with intensive service needs, these patients have wider geographic and clinical location use than most clinic populations due to the transient nature of patients with housing insecurity. What is encouraging, however, is that through this systemwide initiative, there is an impetus to screen veterans, regardless of who performs the screening. This is particularly meaningful given that rates of depression screening may be as low as 4% among PCPs.15 During implementation, the QI team learned that nearly 18% of the empaneled HPACT patients were exempt from screening. The exempt patients do not have an active clinical reminder for depression screens. Instead, these patients are receiving mental health surveillance and specialty treatment, during which continuous monitoring and assessment for suicidal ideation and risk of suicide are performed. Additionally, an EHR-based factor that also may limit appropriate follow-up and contribute to missed opportunities is that secondary and tertiary screens do not populate until the EHR was refreshed after positive primary screens, which introduces human error in a process that could be automated. Both RNs and PCPs may occasionally miss secondary and tertiary screens due to this issue, which continues to be a barrier. Given the high risk HPACT clinic population, the QI team encouraged staff members to frequently screen patients for suicidal ideation regardless of clinical reminders. A consideration for the future would be to identify optimal frequency for screening and to continue to validate assessment methods.
Finally, while the percentage of patients who were considered missed opportunities (visited the HPACT clinic but were not screened) was relatively small at 6% of the total panel of patients, this number theoretically should be zero. Though this project was not designed to identify the specific causes for missed opportunities, future QI efforts may consider evaluating for other potential reasons. These may include differing process flows for various encounters (same-day care visits, scheduled primary care visit, RN-only visit), screening not activating at time of visit, time constraints, or other unseen reasons. Another important population is the 11% of patients who were otherwise eligible for screening but did not visit the HPACT clinic, and in some cases, no other VA location. There are a few explanatory reasons centered on the mobility of this population between health systems. However, this patient population also may be among the most vulnerable and at risk: 62% of veteran suicides in 2017 had not had a VA encounter that year.13 While there is no requirement that the veteran visit the HPACT clinic annually, future efforts may focus on increasing engagement through other means of outreach, including site visits and community care involvement, knowing the nature of the sporadic follow-up patterns in this patient population. Future work may also involve examining suicide rates by primary care clinic and triage patterns between interprofessional staff.
Limitations
Due to the limited sample size, findings cannot be generalized to all VA sites. The QI team used retrospective, administrative data. Additionally, since this is a primary care clinic focused on a specialized population, this result may not be generalizable to all primary care settings, other primary care populations, or even other homeless primary care clinics, though it may establish a benchmark when other clinics internally examine their data and processes.
Conclusions
Improving screening protocols can lead to identification of at-risk individuals who would not have otherwise been identified.16,17 As the US continues to grapple with mental health and suicide, efforts toward addressing this important issue among veterans remains a top priority.
Acknowledgments
Thank you to the VAGLAHS Center of Excellence in Primary Care Education faculty and trainees, the HPACT staff, and the VA Informatics and Computing Infrastructure (VINCI) for data support.
1. Centers for Disease Control and Prevention. Facts about suicide. Reviewed August 30, 2021. Accessed December 13, 2021. https://www.cdc.gov/suicide/facts/index.html
2. Centers for Disease Control and Prevention. Preventing suicide: a technical package of policies, programs, and practices. Published 2017. Accessed December 13, 2021. https://www.cdc.gov/violenceprevention/pdf/suicideTechnicalPackage.pdf
3. Centers for Disease Control and Prevention. Increase in suicide mortality in the United States, 1999-2018. April 8, 2020. Accessed December 13, 2021. https://www.cdc.gov/nchs/products/databriefs/db362.htm
4. US Department of Veterans Affairs, Office of Mental Health and Suicide Prevention. 2020 National Veteran Suicide Prevention Annual Report. Published November 2020. Accessed December 13, 2021. https://www.mentalhealth.va.gov/docs/data-sheets/2020/2020-National-Veteran-Suicide-Prevention-Annual-Report-11-2020-508.pdf
5. Culhane D, Szymkowiak D, Schinka, JA. Suicidality and the onset of homelessness: evidence for a temporal association from VHA treatment records. Psychiatr Serv. 2019;70(11):1049-1052. doi:10.1176/appi.ps.201800415
6. US Department of Housing and Urban Development. The 2015 annual homeless assessment report (AHAR) to Congress. Published November 2015. Accessed December 13, 2021. https://www.hudexchange.info/resources/documents/2015-AHAR-Part-1.pdf
7. US Department of Veterans Affairs, Office of Suicide Prevention. Suicide among veterans and other Americans 2001-2014. Published August 3, 2016. Updated August 2017. Accessed December 13, 2021. https://www.mentalhealth.va.gov/docs/2016suicidedatareport.pdf
8. Dobscha SK, Corson K, Helmer DA, et al. Brief assessment for suicidal ideation in OEF/OIF veterans with positive depression screens. Gen Hosp Psychiatry. 2013;35(3):272-278. doi:10.1016/j.genhosppsych.2012.12.001
9. Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care providers before suicide: a review of the evidence. Am J Psychiatry. 2002;159(6):909-916. doi:10.1176/appi.ajp.159.6.909
10. US Department of Veterans Affairs. National strategy for preventing veteran suicide 2018-2028. Accessed December 13, 2021. https://sprc.org/sites/default/files/resource-program/VA_National-Strategy-for-Preventing-Veterans-Suicide2018.pdf
11. US Department of Veterans Affairs. VA suicide prevention efforts. Published July 2019. Accessed December 15, 2021. https://www.mentalhealth.va.gov/suicide_prevention/docs/VA_Suicide_Prevention_Program_Fact_Sheet_508.pdf
12. Wortzel H, Matarazzo B, Homaifer B. A model for therapeutic risk management of the suicidal patient. J Psychiatr Pract. 2013;19(4):323-326. doi:10.1097/01.pra.0000432603.99211.e8
13. US Department of Veterans Affairs. VA/DoD clinical practice guidelines for the assessment and management of patients at risk for suicide. Provider summary version 2.0. Published 2019. Accessed on December 3, 2020. https://www.healthquality.va.gov/guidelines/MH/srb/VADoDSuicideRiskFullCPGFinal5088919.pdf
14. Bahraini N, Brenner LA, Barry C, et al. Assessment of rates of suicide risk screening and prevalence of positive screening results among US veterans after implementation of the Veterans Affairs suicide risk identification strategy. JAMA Netw Open. 2020;3(10):e2022531. doi:10.1001/jamanetworkopen.2020.22531
15. Akincigil A, Matthews EB. National rates and patterns of depression screening in primary care: results from 2012 and 2013. Psychiatr Serv. 2017;68(7):660-666. doi:10.1176/appi.ps.201600096
16. Posner K, Brent D, Lucas C, et al. Columbia-suicide severity rating scale (C-SSRS). Columbia University Medical Center, New York, NY. 2008. Accessed December 3, 2020. https://cssrs.columbia.edu/wp-content/uploads/C-SSRS-Screening_AU5.1_eng-USori.pdf
17. Boudreaux ED, Camargo CA Jr, Arias SA, et al. Improving suicide risk screening and detection in the emergency department. Am J Prev Med. 2016;50(4):445-453. doi:10.1016/j/amepre.2015.09.029
1. Centers for Disease Control and Prevention. Facts about suicide. Reviewed August 30, 2021. Accessed December 13, 2021. https://www.cdc.gov/suicide/facts/index.html
2. Centers for Disease Control and Prevention. Preventing suicide: a technical package of policies, programs, and practices. Published 2017. Accessed December 13, 2021. https://www.cdc.gov/violenceprevention/pdf/suicideTechnicalPackage.pdf
3. Centers for Disease Control and Prevention. Increase in suicide mortality in the United States, 1999-2018. April 8, 2020. Accessed December 13, 2021. https://www.cdc.gov/nchs/products/databriefs/db362.htm
4. US Department of Veterans Affairs, Office of Mental Health and Suicide Prevention. 2020 National Veteran Suicide Prevention Annual Report. Published November 2020. Accessed December 13, 2021. https://www.mentalhealth.va.gov/docs/data-sheets/2020/2020-National-Veteran-Suicide-Prevention-Annual-Report-11-2020-508.pdf
5. Culhane D, Szymkowiak D, Schinka, JA. Suicidality and the onset of homelessness: evidence for a temporal association from VHA treatment records. Psychiatr Serv. 2019;70(11):1049-1052. doi:10.1176/appi.ps.201800415
6. US Department of Housing and Urban Development. The 2015 annual homeless assessment report (AHAR) to Congress. Published November 2015. Accessed December 13, 2021. https://www.hudexchange.info/resources/documents/2015-AHAR-Part-1.pdf
7. US Department of Veterans Affairs, Office of Suicide Prevention. Suicide among veterans and other Americans 2001-2014. Published August 3, 2016. Updated August 2017. Accessed December 13, 2021. https://www.mentalhealth.va.gov/docs/2016suicidedatareport.pdf
8. Dobscha SK, Corson K, Helmer DA, et al. Brief assessment for suicidal ideation in OEF/OIF veterans with positive depression screens. Gen Hosp Psychiatry. 2013;35(3):272-278. doi:10.1016/j.genhosppsych.2012.12.001
9. Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care providers before suicide: a review of the evidence. Am J Psychiatry. 2002;159(6):909-916. doi:10.1176/appi.ajp.159.6.909
10. US Department of Veterans Affairs. National strategy for preventing veteran suicide 2018-2028. Accessed December 13, 2021. https://sprc.org/sites/default/files/resource-program/VA_National-Strategy-for-Preventing-Veterans-Suicide2018.pdf
11. US Department of Veterans Affairs. VA suicide prevention efforts. Published July 2019. Accessed December 15, 2021. https://www.mentalhealth.va.gov/suicide_prevention/docs/VA_Suicide_Prevention_Program_Fact_Sheet_508.pdf
12. Wortzel H, Matarazzo B, Homaifer B. A model for therapeutic risk management of the suicidal patient. J Psychiatr Pract. 2013;19(4):323-326. doi:10.1097/01.pra.0000432603.99211.e8
13. US Department of Veterans Affairs. VA/DoD clinical practice guidelines for the assessment and management of patients at risk for suicide. Provider summary version 2.0. Published 2019. Accessed on December 3, 2020. https://www.healthquality.va.gov/guidelines/MH/srb/VADoDSuicideRiskFullCPGFinal5088919.pdf
14. Bahraini N, Brenner LA, Barry C, et al. Assessment of rates of suicide risk screening and prevalence of positive screening results among US veterans after implementation of the Veterans Affairs suicide risk identification strategy. JAMA Netw Open. 2020;3(10):e2022531. doi:10.1001/jamanetworkopen.2020.22531
15. Akincigil A, Matthews EB. National rates and patterns of depression screening in primary care: results from 2012 and 2013. Psychiatr Serv. 2017;68(7):660-666. doi:10.1176/appi.ps.201600096
16. Posner K, Brent D, Lucas C, et al. Columbia-suicide severity rating scale (C-SSRS). Columbia University Medical Center, New York, NY. 2008. Accessed December 3, 2020. https://cssrs.columbia.edu/wp-content/uploads/C-SSRS-Screening_AU5.1_eng-USori.pdf
17. Boudreaux ED, Camargo CA Jr, Arias SA, et al. Improving suicide risk screening and detection in the emergency department. Am J Prev Med. 2016;50(4):445-453. doi:10.1016/j/amepre.2015.09.029