The reliability of self-report assessments of the australian treatment outcome profile and world health organization's alcohol, smoking, and substance involvement screening test compared with standard interviews in residential treatment for substance use disorders

Abstract

Introduction and Aims: Alcohol and other drug treatment services require cost-effective, time-efficient assessments. Two widely used assessments are the Australian Treatment Outcome Profile (ATOP), and the World Health Organization's Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST). The present study assessed their psychometric properties regarding reliable self-reporting within residential treatment facilities.

Design and Methods: At total of 294 participants completed a self-report retrospective assessment of substance use, from the point of cessation of substance use prior to entry to residential treatment: Online assessment of the ATOP (for the previous 30 days) and ASSIST (for the previous 90 days). This was repeated between one to seven days later in a face-to-face interview (N=104).

Results: Internal consistency for the ASSIST was moderate to high (Cronbach's alpha >0.60) for both self-report and interview-administered versions, but not applicable for the ATOP. There was strong concordance within measures at both time points across most substances (intraclass correlation coefficient >0.74 ATOP and >0.76 ASSIST). Poor concordance (intraclass correlation coefficient <0.56) was found for other opiates in the ATOP and tobacco use in the ASSIST. The level of agreement between the two measures was moderate to strong (Kappa >0.60) for most substances for the ATOP except alcohol (0.15) and cannabis (0.58). Conversely, only two measures of the ASSIST were moderate: heroin (0.74) and injecting practices (0.68). The remainder of substances reflecting weak agreement (Kappa <0.57), and three (alcohol, tobacco and cannabis) reflecting poor agreement (Kappa=0.09-0.25). The number of days since cessation of use and date of completion of online self-report was significantly associated with the reliability of individual reporting. Poor literacy skills were significantly related to variability in reporting opioids.

Discussions and Conclusions: Reliability of self-reported online assessments of alcohol and other drug use using the ATOP and ASSIST were comparable to face-to-face interviews for most substance types, and supports their use for intake and research. An important consideration should be length of time since cessation of use for reliable recall.

Implications for Practice: These findings provide confidence in the use of online completion of valid and reliable measures of substance use, contributing to reduction in assessment time and burden for treatment services.