Skip to main content

Assessing success—a commentary on the necessity of outcomes measures

Abstract

Measurements for outcomes reporting are not fully formed and utilized in the American addiction industry, though formulated and adopted elsewhere in the world. While studies have established demographic information about those needing and receiving treatment as well as the facilities that offer such treatment, short- and long-term outcomes are scantily reported. This commentary serves as a call to action to developing such metrics in the US by illustrating the benefits to treatment providers and clients of creating outcomes standards, and the subsequent improvements in quality of care needed to reach those standards. Benefits of developing these metrics beyond improved quality of care may also include a more efficient allocation of resources, such as time and money. Additionally, the delivery of more effective, personalized, and outcomes-driven addiction treatment may increase client buy-in and foster a more open communication channel between clients and providers during and after treatment.

Background

Recent data from the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) National Survey on Drug Use and Health [1] (NSDUH) indicates that 23.5 million Americans aged 12 or older (approximately 9.4 % of the American population over age 12) are current illicit drug users. Of these illicit drug users, 11.2 % receive treatment for substance use disorders and addiction [1]. According to SAMHSA’s National Survey of Substance Abuse Treatment Services [2] (N-SSATS), 14,311 addiction treatment facilities or centers in the US provided care for 1.25 million clients in 2012. 54.9 % of admitted cases, in one national survey, have had at least one prior treatment episode [3]. While demographic data about those struggling with addiction are readily reported [1, 3, 4], less information is available on those who have completed or exited a treatment facility. SAMHSA’s Treatment Episodes Data Set [3] (TEDS) reports on number of prior treatment episodes upon entry into a treatment center; however indicators of treatment effectiveness and follow-up data are not explicitly and regularly tracked on a national level. Furthermore, TEDS (and other) surveys do not make unique cases publicly available further highlighting this lack of information, as well as exposing a missed opportunity to qualitatively evaluate individualized treatment plans.

Main text

Standardized tools for addiction professionals to measure short- and long-term outcomes and success, across multiple metrics, are not readily available in the US. In fact, the definition of “success” as it pertains to recovery is, at best, vaguely defined. For some, including 12-Step adherents, “success” is complete abstinence from any substance, whereas others may consider “success” to include the use of pharmacotherapies to manage symptoms and cravings, or employing techniques to use a particular substance responsibly [5]. Further still, the concept of “recovery” is ill-defined, and is generally poorly understood [6]. Peer reviewed studies on “success” and outcomes lack universal consensus for the American addiction treatment system [7], and are limited in their practical application. For example, the prominent TEDS-D survey defines “success” solely as completing a treatment program. While more comprehensive outcomes measures have been investigated on facility [8] and state [9, 10] levels, long-term, multi-metric measures on a national scale have seldom been investigated, particularly with support, financial or otherwise, from governing bodies. Analogous metrics have, however, been created for addiction treatment systems elsewhere. The European Union, with support from the United Nations, has published extensive survey tools and guidelines for addiction treatment providers to measure and assess outcomes [1113]. Similarly, the Australian government provides metrics and compulsory standards, as well as toolkits containing guidelines, resources, and educational materials [14].

Relapse rate has long been the standard indicator by which recovery success is measured within the industry. Historically though, documentation of this metric has been inconsistent, and has relied heavily on estimated figures. Estimates suggest that as many as 80 % of those who seek and complete addiction treatment will relapse after treatment termination [15]. These grim statistics suggest, perhaps, that relapse rate cannot serve as the sole indicator of recovery success, lest the industry as a whole be interpreted as widely ineffectual. Addiction treatment and recovery are multifaceted processes, and treatment success cannot be determined by a single metric. While relapse rates may indicate some facet of success, other metrics, such as comorbidities; personal, legal, and emotional state; and external conditions, may prove significant to long-term recovery success and should be considered as potential metrics to determining a composite success variable [16].

The currently available data on post-rehabilitation outcomes is inconsistent, and demonstrates a lack of reporting in the addiction space, which can contribute to two major consequences. First, without accurate outcomes evaluation, facilities offering poor quality of care are able to continue offering such treatment without regular evaluation. This absence of measurements and tracking also results in a poor allocation of resources: facilities offering poor treatment quality continue to consume resources that would otherwise be better spent by facilities engaging in ethical, high quality treatment. Secondly, without outcomes measurements, there are fewer options for conducting quality improvement initiatives, both on the facility level and on a broad level throughout the space. This stagnates the industry, as there are no quantifiable standards to achieve and surpass; rather, progress becomes more ideological and thus, harder to define. To motivate constant quality improvement, measurable outcomes must be assessed and publicly reported.

Outcomes are directly dependent on the level of care received throughout treatment. By setting outcomes standards, the quality of treatment may elevate, and the form of treatment delivery may change to meet these standards. Dialogue on treatment methodologies is widespread in the addiction space, often focused on which particular treatment method is the most effective. Following the evidence borne from current trends in the broader American healthcare system, this dialogue should include personalized treatment plans [17, 18]. Designing and implementing outcomes reporting, based on leading treatment methodologies, would, in effect, install those methodologies into facilities’ practices.

An outcomes reporting system that incorporates leading treatment methodologies could have profound effects on the treatment space. Through delivering more effective outcomes-driven treatment, with standards that allow for individualization, clients may engage in building a treatment plan that is best suited to their needs, preferences, and belief system [17]. This possibility may increase buy-in from clients, who would otherwise be resistant to receiving treatment without a more trust-founded and personal connection to the treatment plan and provider – ultimately culminating in increased treatment adherence, setting the client up for long-term success [1720]. By involving clients in the development of their own treatment plan, clients may feel more empowered and profoundly engaged with the recovery process. The clients become active participants, rather than resigned recipients of a rote prescription for an immutable treatment process.

The ways in which outcomes are measured may also benefit post-treatment client buy-in. Previous clients, who struggle or relapse after receiving treatment, may be difficult to reach for follow-up. These past clients may feel guilt, shame, or defeat, thus prompting them to distance themselves from their previous treatment facility [21]. A personal treatment plan may make the client feel more connected to the treatment provider; therefore, opening the communication channel between alumni and treatment centers to follow-up or receive additional help.

Increasing treatment effectiveness also requires assessing aspects of treatment beyond what is considered clinical. Administrative assessments must be conducted to determine which services a treatment provider is equipped to provide. Attention must be drawn to facilities’ physical accommodations and the abilities of the staff, including credentials, legal limitations, and staff-to-client ratios. Accurate and truthful assessments of these capabilities will allow for facilities to provide only the treatment plans they can manage, rather than overextending themselves and causing unnecessary expenditures. Treatment providers must also consider who is an ideal client for their center – one whose needs align best with the providers’ missions, skills, and services. Ideal matches allow for the right treatment to be delivered at the right time with the right resources to the right people. Doing so allows for a much more efficient allocation of resources, which saves clients and treatment providers time, money, and personnel.

Conclusion

Ultimately, more effective treatment driven by smarter decision-making and outcomes standards has the capability to benefit clients and their loved ones, as well as facilities themselves. In addition to the time, money, and other resources saved for both clients and facilities, facilities have the added benefit of using outcomes measures for accreditation, marketing materials, and their reputation in the space. A multimetric system for evaluating outcomes must be developed to motivate enhanced treatment effectiveness, cyclically benefiting individuals and facilities in the long-term. Future projects should identify these metrics using those established in the European Union and Australia as guides, and develop a standardized process by which outcomes can be measured and reported.

Abbreviations

SAMHSA:

Substance abuse and mental health services administration

NSDUH:

National survey on drug use and health

N-SSATS:

National survey of substance abuse treatment services

TEDS:

Treatment episodes data set

References

  1. Substance Abuse and Mental Health Services Administration. Results from the 2009 National Survey on Drug Use and Health: Volume I. Summary of National Findings. Office of Applied Studies, NSDUH Series H-38A, HHS Publication No. SMA 10–4856 Findings. Rockville, MD: US Department of Health and Human Services; 2010.

    Google Scholar 

  2. United States Department of Health and Human Services. Substance Abuse and Mental Health Services Administration. Center for Behavioral Health Statistics and Quality: National Survey of Substance Abuse Treatment Services (N-SSATS): 2012. Data on Substance Abuse Treatment Facilities. Rockville, Maryland: US Department of Health and Human Services; 2013.

    Google Scholar 

  3. United States Department of Health and Human Services. Substance Abuse and Mental Health Services Administration. Center for Behavioral Health Statistics and Quality: Treatment Episode Data Set—Admissions (TEDS-A) 2012. Rockville, MD: US Department of Health and Human Services; 2014.

    Google Scholar 

  4. United States Department of Health and Human Services. Substance Abuse and Mental Health Services Administration. Center for Behavioral Health Statistics and Quality: Treatment Episode Data Set—Discharges (TEDS-D), 2011. Rockville, MD: US Department of Health and Human Services; 2014.

    Google Scholar 

  5. Cherkis J. Dying To Be Free. Washington DC: The Huffington Post; 2015.

    Google Scholar 

  6. Laudet AB. What does recovery mean to you? Lessons from the recovery experience for research and practice. J Subst Abuse Treat. 2007;33(3):243–56.

    Article  PubMed Central  PubMed  Google Scholar 

  7. Graham K. Guidelines for using standardized outcome measures following addictions treatment. Evaluation & the Health Professions. 1994;17:43–59.

    Article  CAS  Google Scholar 

  8. Outcomes of Alcohol/Other Drug Dependency Treatment. Butler Center for Research: Research Update. Center City, MN: The Hazelden Betty Ford Foundation; 2011. p. 2.

    Google Scholar 

  9. Falkowski CL. Chemical Dependency Provider Performance Measures. St. Paul, MN: Minnesota Department of Health Services; 2013. p. 803.

    Google Scholar 

  10. Hedden S, Guard M, Arndt S. State of Iowa Outcomes Monitoring System: Evaluation Trend Report. Iowa Department of Public Health Contract #5881NA01. Iowa City, IA: The Iowa Consortium for Substance Abuse Research and Evaluation; 2011.

    Google Scholar 

  11. Schippers G, Broekman T. Measurements in the Addictions for Triage and Evaluation – MATE 2.1 Manual and Protocol. English Edition. Bureau Beta, Nijmegen, The Netherlands: European Monitoring Centre for Drugs and Drug Addiction; 2003.

    Google Scholar 

  12. Donmall M, Jones A, Lawrinson P, Long J, Millar T, Royuela Morales L, et al. Guidance for the measurement of drug treatment demand. New York, NY: Global Assessment Programme on Drug Abuse in collaboration with the European Monitoring Centre for Drugs and Drug Addiction; 2006. p. 58.

    Google Scholar 

  13. DeLeon G. CMRS Scales for Substance Abuse Treatment. Center for Therapeutic Community Research (CTCR). New York, NY: European Monitoring Centre for Drugs and Drug Addictions; 1993.

    Google Scholar 

  14. Deady M. Summary Table of Assessment and Outcomes Measure Tools. Network of Alcohol & Other Drugs Agencies (NADA) as part of the Drug and Alcohol and Mental Health Information Management Project. 2009:14.

  15. Marlatt GA, Gordon JR, editors. Relapse Prevention: Maintenance Strategies in the Treatment of Addictive Behaviors. New York: Guilford Press; 2005. p. 416.

    Google Scholar 

  16. Teesson M, Clement N, Copeland J, Conroy A, Reid A. The Measurement of Outcome in Alcohol and Other Drug Treatment: A Review of Available Instruments. National Drug and Alcohol Research Centre (NDARC) Technical Report No. 90. Australia: University of New South Wales; 2000.

    Google Scholar 

  17. Drake RE, Cimpean D, Torrey WC. Shared decision making in mental health: prospects for personalized medicine. Dialogues in Clinical Neurosciences. 2009;11:455–63.

    Google Scholar 

  18. Sacristán JA. Patient-centered medicine and patient-oriented research: improving health outcomes for individual patients. BMC Med Informat Decis Making. 2013;13:6.

    Article  Google Scholar 

  19. Di Paula A, Long R, Wiener DE. Are your patients satisfied? Market Health Serv. 2002;22:28–32.

    Google Scholar 

  20. Vermeire E, Hearnshaw H, Van Royen P, Denekens J. Patient adherence to treatment: three decades of research. A comprehensive review. J Clin Pharm Therapeut. 2001;26:331–42.

    Article  CAS  Google Scholar 

  21. O’Connor LE, Berry JW, Inaba D, Weiss J, Morrison A. Shame, guilt, and depression in men and women in recovery from addiction. J Subst Abuse Treat. 1994;11:503–10.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors wish to acknowledge Jeff Smith, Abhilash Patel, and Melanie Haber for their support of this publication, as well as Dr. Scot Thomas for his editorial contributions.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ruchi M. Sanghani.

Additional information

Competing interests

The Coalition Against Drug Abuse is affiliated with Recovery Brands LLC. Recovery Brands LLC has financially supported this paper.

Authors’ contributions

RMS and AC conducted literature reviews. RMS, AC, and AKM planned and prepared this commentary. All listed authors have given final approval of this document, and agree to be accountable for all aspects of the work.

Authors’ information

RMS holds a Master’s degree in Bioethics, with a concentration on health care research ethics. Previous works conducted by RMS include physician-industry transparency research at The Cleveland Clinic Department of Bioethics, and conflict of interest reporting at the Edmond J. Safra Center for Ethics at Harvard University, where she served as a Research Ethics Fellow. AKM holds a Master’s degree in Political Science, with experience conducting health policy research in an international context, including projects contracted by the United Nations and USAID. AC is a student in the Psychology Department at Fordham University, specializing in addiction treatment.

Rights and permissions

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sanghani, R.M., Carlin, A.L. & Moler, A.K. Assessing success—a commentary on the necessity of outcomes measures. Subst Abuse Treat Prev Policy 10, 20 (2015). https://0-doi-org.brum.beds.ac.uk/10.1186/s13011-015-0017-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13011-015-0017-2

Keywords