Holmes, E. A. et al. The Lancet Psychiatry Commission on psychological treatments research in tomorrow’s science. Lancet Psychiatry. 5, 237–286 (2018).
Google Scholar
Lambert, M. J. Outcome in psychotherapy: the past and important advances. Psychotherapy. 50, 42–51 (2013).
Google Scholar
Cuijpers, P., van Straten, A., Andersson, G. & van Oppen, P. Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies. J. Consult. Clin. Psychol. 76, 909–922 (2008).
Google Scholar
Barth, J. et al. Comparative efficacy of seven psychotherapeutic interventions for patients with depression: a network meta-analysis. PLoS Med. 10, e1001454 (2013).
Google Scholar
Arch, J. J. et al. Randomized clinical trial of cognitive behavioral therapy (CBT) versus acceptance and commitment therapy (ACT) for mixed anxiety disorders. J. Consult. Clin. Psychol. 80, 750–765 (2012).
Google Scholar
Bögels, S. M., Wijts, P., Oort, F. J. & Sallaerts, S. J. M. Psychodynamic psychotherapy versus cognitive behavior therapy for social anxiety disorder: an efficacy and partial effectiveness trial. Depress Anxiety. 31, 363–373 (2014).
Google Scholar
Markowitz, J. C. et al. Is Exposure Necessary? A Randomized Clinical Trial of Interpersonal Psychotherapy for PTSD. Am J. Psychiatry. 172, 430–440 (2015).
Google Scholar
Wampold, B. E. & Brown, G. S. J. Estimating variability in outcomes attributable to therapists: a naturalistic study of outcomes in managed care. J. Consult. Clin. Psychol. 73, 914–923 (2005).
Google Scholar
Lambert, M. J. ed. Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change. John Wiley & Sons; 2013.
Barkham, M., Lutz, W., Lambert, M. J. & Saxon, D. Therapist effects, effective therapists, and the law of variability. In Castonguay L. G., Hill C. E., eds. How and Why Are Some Therapists Better than Others?: Understanding Therapist Effects. Vol 356. American Psychological Association, xv; 2017:13–36.
Owen, J. et al. Are high-performing therapists both effective and consistent? A test of therapist expertise. J. Consult. Clin. Psychol. 87, 1149–1156 (2019).
Google Scholar
Erekson, D. M., Clayson, R., Park, S. Y. & Tass, S. Therapist effects on early change in psychotherapy in a naturalistic setting. Psychother Res. 30, 68–78 (2020).
Google Scholar
Wampold, B. E. & Owen, J. Therapist effects: History, methods, magnitude. In: Barkham M., Lutz W., Castonguary L. G., eds. Bergin and Garfield’s Handbook of Psychotherapy. John Wiley & Sons, Inc.; 2021:297–326.
Castonguay, L. G., Boswell, J. F., Constantino, M. J., Goldfried, M. R. & Hill, C. E. Training implications of harmful effects of psychological treatments. Am Psychol. 65, 34–49 (2010).
Google Scholar
Elliott, R. Psychotherapy change process research: realizing the promise. Psychother Res. 20, 123–135 (2010).
Google Scholar
Goldfried, M. R. & Wolfe, B. E. Toward a more clinically valid approach to therapy research. J. Consult. Clin. Psychol. 66, 143–150 (1998).
Google Scholar
Stiles, W. B. Verbal response modes and psychotherapeutic technique. Psychiatry. 42, 49–62 (1979).
Google Scholar
Benjamin, L. S. Structural analysis of social behavior. Psychol. Rev. 81, 392–425 (1974).
Google Scholar
Flemotomos, N. et al. Automated evaluation of psychotherapy skills using speech and language technologies. Behav Res Methods. Published online August 3, 2021. https://doi.org/10.3758/s13428-021-01623-4.
Rogers, C. R. The use of electrically recorded interviews in improving psychotherapeutic techniques. Am. J. Orthopsychiatry. 12, 429–434 (1942).
Google Scholar
Hofmann, S. G. & Hayes, S. C. The future of intervention science: process-based therapy. Clin. Psychol. Sci. 7, 37–50 (2019).
Google Scholar
Miner, A. S. et al. Assessing the accuracy of automatic speech recognition for psychotherapy. NPJ Digit. Med. 3, 82 (2020).
Google Scholar
Goldfried, M. R. Obtaining consensus in psychotherapy: What holds us back? Am. Psychol. 74, 484–496 (2019).
Google Scholar
Kazdin, A. E. Addressing the treatment gap: A key challenge for extending evidence-based psychosocial interventions. Behav. Res. Ther. 88, 7–18 (2017).
Google Scholar
Hill, C. E. & Lent, R. W. A narrative and meta-analytic review of helping skills training: Time to revive a dormant area of inquiry. Gaines AN, Goldfried MR, Constantino MJ. Revived call for consensus in the future of psychotherapy. Evid. Based. Ment. Health. 24, 2–4 (2021).
Mehta, M., et al. Psychotherapy is Not One Thing: Simultaneous Modeling of Different Therapeutic Approaches. In Proc. the Eighth Workshop on Computational Linguistics and Clinical Psychology. Association for Computational Linguistics; 2022:47–58.
Lee, M. & Martin, J. L. Coding, counting and cultural cartography. Am. J. Cultural Sociology. 3, 1–33 (2015).
Google Scholar
Goldberg, S. B. et al. Machine learning and natural language processing in psychotherapy research: Alliance as example use case. J. Couns. Psychol. 67, 438–448 (2020).
Google Scholar
Gaines, A. N., Goldfried, M. R. & Constantino, M. J. Revived call for consensus in the future of psychotherapy. Evid Based Ment. Health. 24, 2–4 (2021).
Google Scholar
Althoff, T., Clark, K., & Leskovec. J. Large-scale analysis of counseling conversations: An application of natural language processing to mental health. In Transactions of the Association for Computational Linguistics, Volume 4.; 2016:463–476.
Eichstaedt, J. C. et al. Closed- and open-vocabulary approaches to text analysis: A review, quantitative comparison, and recommendations. Psychol. Methods. 26, 398–427 (2021).
Google Scholar
Imel, Z. E., Steyvers, M. & Atkins, D. C. Computational psychotherapy research: Scaling up the evaluation of patient-provider interactions. Psychotherapy. 52, 19–30 (2015).
Google Scholar
Malhotra, G., Waheed, A., Srivastava, A., Akhtar, M. S., & Chakraborty, T. Speaker and time-aware joint contextual learning for dialogue-act classification in counselling conversations. In Proc. the Fifteenth ACM International Conference on Web Search and Data Mining.; 2021:735–745.
Doorn, K. A. van, Kamsteeg, C., Bate, J. & Aafjes. M. A scoping review of machine learning in psychotherapy research. Psychother Res. Published online August 29, 2020. https://doi.org/10.1080/10503307.2020.1808729.
Xiao, B., Imel, Z. E., Atkins, D. C., Georgiou, P. G., Narayanan, S. S. Analyzing speech rate entrainment and its relation to therapist empathy in drug addiction counseling. In Proc. Interspeech; 2015:2489–2493.
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C. & Althoff T. Towards facilitating empathic conversations in online mental health support: a reinforcement learning approach. In Proc. of the Web Conference 2021. WWW ’21. (Association for Computing Machinery, 2021:194–205).
Nook, E. C., Vidal Bustamante, C. M., Cho, H. Y. & Somerville, L. H. Use of linguistic distancing and cognitive reappraisal strategies during emotion regulation in children, adolescents, and young adults. Emotion. 20, 525–540 (2020).
Google Scholar
Lee, F. T., Hull, D., Levine, J., Ray, B. & McKeown, K. Identifying therapist conversational actions across diverse psychotherapeutic approaches. In Proc. of the Sixth Workshop on Computational Linguistics and Clinical Psychology. (Association for Computational Linguistics, 2019:12–23).
Ewbank, M. P. et al. Quantifying the association between psychotherapy content and clinical outcomes using deep learning. JAMA Psychiatry. 77, 35–43 (2020).
Google Scholar
Roth, A. & Fonagy P. What Works for Whom?: A Critical Review of Psychotherapy Research. (Guilford Press, 2006).
Castonguay, L. G. & Hill, C. E., (eds) How and Why Are Some Therapists Better than Others?: Understanding Therapist Effects. Vol 356. (American Psychological Association, 2017).
Lattie, E. G., Stiles-Shields, C. & Graham, A. K. An overview of and recommendations for more accessible digital mental health services. Nat. Rev. Psychol. 26, 1–14 (2022).
Haque, A., Milstein, A. & Fei-Fei, L. Illuminating the dark spaces of healthcare with ambient intelligence. Nature 585, 193–202 (2020).
Google Scholar
Tausczik, Y. R. & Pennebaker, J. W. The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods. J. Lang Soc. Psychol. 29, 24–54 (2010).
Google Scholar
Zimmermann, J., Brockmeyer, T., Hunn, M., Schauenburg, H. & Wolf, M. First-person Pronoun Use in Spoken Language as a Predictor of Future Depressive Symptoms: Preliminary Evidence from a Clinical Sample of Depressed Patients. Clin. Psychol. Psychother. 24, 384–391 (2017).
Google Scholar
Vine, V., Boyd, R. L. & Pennebaker, J. W. Natural emotion vocabularies as windows on distress and well-being. Nat. Commun. 11, 4525 (2020).
Google Scholar
Wadden, D., August, T., Li, Q. & Althoff, T. The effect of moderation on online mental health conversations. In Proc. of the Fifteenth International AAAI Conference on Web and Social Media (ICWSM 2021).; 2021:751–763.
Beck, J. S. Cognitive Behavior Therapy, Second Edition: Basics and Beyond. (Guilford Press, 2011).
Weissman, M. M., Markowitz, J. C., Klerman, G. L. Clinician’s Quick Guide to Interpersonal Psychotherapy. Vol 165; 2008:140–141.
Baird, H. M., Webb, T. L., Sirois, F. M. & Gibson-Miller, J. Understanding the effects of time perspective: A meta-analysis testing a self-regulatory framework. Psychol. Bull. 147, 233–267 (2021).
Google Scholar
Park, G. et al. Living in the Past, Present, and Future: Measuring Temporal Orientation With Language. J .Pers. 85, 270–280 (2017).
Google Scholar
Keefe, J. R. et al. In-session emotional expression predicts symptomatic and panic-specific reflective functioning improvements in panic-focused psychodynamic psychotherapy. Gross JJ, Jazaieri H. Emotion, Emotion Regulation, and Psychopathology: An Affective Science Perspective. Clin. Psychol. Sci. 2, 387–401 (2014).
Nook, E. C., Schleider, J. L. & Somerville, L. H. A linguistic signature of psychological distancing in emotion regulation. J. Exp. Psychol. Gen. 146, 337–346 (2017).
Google Scholar
Greenberg, L. S. Emotions, the great captains of our lives: their role in the process of change in psychotherapy. Am. Psychol. 67, 697–707 (2012).
Google Scholar
Fiske, A. P. The lexical fallacy in emotion research: Mistaking vernacular words for psychological entities. Psychol. Rev. 127, 95–113 (2020).
Google Scholar
Gross, J. J. & Jazaieri, H. Emotion, Emotion Regulation, and Psychopathology: An Affective Science Perspective. Clin. Psychol. Sci. 2, 387–401 (2014).
Google Scholar
Greenberg, L. S. & Safran, J. D. Emotion in psychotherapy. Am. Psychol. 44, 19–29 (1989).
Google Scholar
Rocco, D. et al. Beyond Verbal Behavior: An Empirical Analysis of Speech Rates in Psychotherapy Sessions. Front Psychol. 9, 978 (2018).
Google Scholar
Tonti, M. & Gelo, O. C. G. Rate of speech and emotional-cognitive regulation in the psychotherapeutic process: a pilot study. Res. Psychother.: Psychopathol. Process and Outcome. 19, 92–102 (2016). https://doi.org/10.4081/ripppo.2016.232.
Barkham, M., Lutz, W. & Castonguay, L. G. Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change. (John Wiley & Sons, 2021).
Wilfley, D. E. et al. Training models for implementing evidence-based psychological treatment: A cluster-randomized trial in college counseling centers. JAMA Psychiatry. 77, 139–147 (2020).
Google Scholar
Hyndman, R. J., Athanasopoulos G. Forecasting: Principles and Practice. (OTexts, 2018).
Labov, W. & Fanshel D. Therapeutic Discourse: Psychotherapy as Conversation. (Academic Press, 1977).
Clark, H. H. Using Language. (Cambridge University Press, 1996).
Borelli, J. L. et al. Therapist-client language matching: initial promise as a measure of therapist-client relationship quality. Psychoanal. Psychol. 36, 9–18 (2019).
Google Scholar
Ireland, M. E. et al. Language style matching predicts relationship initiation and stability. Psychol. Sci. 22, 39–44 (2011).
Google Scholar
Brinberg, M. & Ram, N. Do New Romantic Couples Use More Similar Language Over Time? Evidence from Intensive Longitudinal Text Messages. J. Commun. 71, 454–477 (2021).
Google Scholar
Koole, S. L. & Tschacher, W. Synchrony in Psychotherapy: A Review and an Integrative Framework for the Therapeutic Alliance. Front Psychol. 2016;7. https://doi.org/10.3389/fpsyg.2016.00862.
Doré, B. P. & Morris, R. R. Linguistic Synchrony Predicts the Immediate and Lasting Impact of Text-Based Emotional Support. Psychol. Sci. 29, 1716–1723 (2018).
Google Scholar
Shapira, N., Atzil-Slonim, D., Tuval Mashiach, R. & Shapira O. Measuring Linguistic Synchrony in Psychotherapy. In: Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology. (Association for Computational Linguistics, 2022:158–176).
Giles, H., Mulac, A., Bradac, J. J. & Johnson, P. Speech accommodation theory: the first decade and beyond. Annals Intern. Commun. Association. 10, 13–48 (1987).
Google Scholar
Hatcher, R. L. Interpersonal competencies: Responsiveness, technique, and training in psychotherapy. Am. Psychol. 70, 747–757 (2015).
Google Scholar
Duran, N. D., Paxton, A. & Fusaroli, R. ALIGN: Analyzing linguistic interactions with generalizable techNiques—A Python library. Psychol. Methods. 24, 419–438 (2019).
Google Scholar
Burkhardt, H. A. et al. Behavioral activation and depression symptomatology: longitudinal assessment of linguistic indicators in text-based therapy sessions. J. Med. Internet Res. 23, e28244 (2021).
Google Scholar
Nook, E. C., Hull, T. D., Nock M. & Somerville L. Linguistic measures of psychological distance track symptom levels and treatment outcomes in a large set of psychotherapy transcripts. https://doi.org/10.31234/osf.io/hqxaz.
Pittenger, R. E., Hockett C. F. & Danehy J. J. The First Five Minutes: A Sample of Microscopic Interview Analysis. (Paul Martineau, 1960).
Horn, R. L. & Weisz, J. R. Can artificial intelligence improve psychotherapy research and practice? Adm. Policy Ment. Health. 47, 852–855 (2020).
Google Scholar
Sharma, A., Miner, A., Atkins, D. & Althoff T. A computational approach to understanding empathy expressed in text-based mental health support. In The 2020 Conference on Empirical Methods in Natural Language Processing.; 2020:5263–5276.
Costa, B. & Dewaele, J. M. Psychotherapy across languages: beliefs, attitudes and practices of monolingual and multilingual therapists with their multilingual patients. Language Psychoanalysis. 1, 18–40 (2012).
Google Scholar
Benish, S. G., Quintana, S. & Wampold, B. E. Culturally adapted psychotherapy and the legitimacy of myth: a direct-comparison meta-analysis. J. Couns. Psychol. 58, 279–289 (2011).
Google Scholar
Whaley, A. L. & Davis, K. E. Cultural competence and evidence-based practice in mental health services: a complementary perspective. Am. Psychol. 62, 563–574 (2007).
Google Scholar
Hook, J. N. et al. Cultural humility and racial microaggressions in counseling. J. Couns. Psychol. 63, 269–277 (2016).
Google Scholar
Koenecke, A. et al. Racial disparities in automated speech recognition. Proc. Natl Acad. Sci. 117, 7684–7689 (2020).
Google Scholar
Corcoran, C. M., Benavides, C. & Cecchi, G. Natural language processing: opportunities and challenges for patients, providers, and hospital systems. Psychiatric Annals. 49, 202–208 (2019).
Google Scholar
Mitchell, M. et al. Diversity and Inclusion Metrics in Subset Selection. In Proc. of the AAAI/ACM Conference on AI, Ethics, and Society. AIES ’20. Association for Computing Machinery; 2020:117-123.
Rubin, E. Striving for Diversity in Research Studies. N. Engl .J. Med. Published online September 13, 2021. https://doi.org/10.1056/NEJMe2114651.
Brown, S., Davidovic, J. & Hasan, A. The algorithm audit: Scoring the algorithms that score us. Big Data & Society. 8, 2053951720983865 (2021).
Google Scholar
Krause, K. R., Chung, S., Sousa Fialho, M. L., Szatmari, P. & Wolpert, M. The challenge of ensuring affordability, sustainability, consistency, and adaptability in the common metrics agenda. Lancet Psychiatry. 8, 1094–1102 (2021).
Google Scholar
Hernandez-Boussard, T., Bozkurt, S., Ioannidis, J. P. A. & Shah, N. H. MINIMAR (MINimum Information for Medical AI Reporting): Developing reporting standards for artificial intelligence in health care. J. Am. Med. Inform. Assoc. 27, 2011–2015 (2020).
Google Scholar
Xiao, B., Imel, Z. E., Georgiou, P. G., Atkins, D. C. & Narayanan, S. S. “Rate My Therapist”: automated detection of empathy in drug and alcohol counseling via speech and language processing. PLoS One. 10, e0143055 (2015).
Google Scholar
Bone, C. et al. Dynamic prediction of psychological treatment outcomes: development and validation of a prediction model using routinely collected symptom data. The Lancet Digital Health. 3, e231–e240 (2021).
Google Scholar
Huckvale, K., Venkatesh, S. & Christensen, H. Toward clinical digital phenotyping: a timely opportunity to consider purpose, quality, and safety. NPJ Digit Med. 2, 88 (2019).
Google Scholar
Jobin, A., Ienca, M. & Vayena, E. The global landscape of AI ethics guidelines. Nat. Machine Intelligence. 1, 389–399 (2019).
Google Scholar
Wilkinson, M. D. et al. The FAIR guiding principles for scientific data management and stewardship. Sci. Data. 3, 160018 (2016).
Google Scholar
Krieger, N. Epidemiology and the People’s Health: Theory and Context. (Oxford University Press, 2011).
Wilfley, D. E. et al. Training models for implementing evidence-based psychological treatment for college mental health: A cluster randomized trial study protocol. Contemp Clin Trials. 72, 117–125 (2018).
Google Scholar
Linstone H. A. & Turoff M. The Delphi Method: Techniques and Applications. First Edition. (Addison-Wesley Educational Publishers Inc, 1975).
Wallach, J. D., Boyack, K. W. & Ioannidis, J. P. A. Reproducible research practices, transparency, and open access data in the biomedical literature, 2015-2017. PLoS Biol. 16, e2006930 (2018).
Google Scholar
Kroenke, K., Spitzer, R. L. & Williams, J. B. W. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 16, 606–613 (2001).
Google Scholar
Stochl, J. et al. On Dimensionality, Measurement Invariance, and Suitability of Sum Scores for the PHQ-9 and the GAD-7. Assessment. 29:355–366. Published online December 3, 2020:1073191120976863.
Zimmerman, M. Using the 9-Item Patient Health Questionnaire to Screen for and Monitor Depression. JAMA. Published online October 18, 2019. https://doi.org/10.1001/jama.2019.15883.
Aafjes-van Doorn, K. & Barber, J. P. Systematic review of in-session affect experience in cognitive behavioral therapy for depression. Cognit. Ther. Res. 41, 807–828 (2017).
Google Scholar
Burkhardt, H., Pullmann, M., Hull, T., Aren, P., Cohen, T. Comparing emotion feature extraction approaches for predicting depression and anxiety. In Proc. of the Eighth Workshop on Computational Linguistics and Clinical Psychology. (Association for Computational Linguistics, 2022:105–115).
Fitzgerald, P. & Leudar, I. On active listening in person-centred, solution-focused psychotherapy. J. Pragmat. 42, 3188–3198 (2010).
Google Scholar
Al-Mosaiwi, M. & Johnstone, T. In an absolute state: elevated use of absolutist words is a marker specific to anxiety, depression, and suicidal ideation. Clin. Psychol. Sci. 6, 529–542 (2018).
Google Scholar
Mehrabian, A. & Ferris, S. R. Inference of attitudes from nonverbal communication in two channels. J. Consult. Psychol. 31, 248–252 (1967).
Google Scholar
Mehrabian, A. & Wiener, M. Decoding of inconsistent communications. J. Personality Social. Psychol. 6, 109–114 (1967).
Google Scholar
Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition. (Springer Science & Business Media, 2009).
Mann, H. B. & Whitney, D. R. On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18, 50–60 (1947).
Google Scholar
Runge, J., Nowack, P., Kretschmer, M., Flaxman, S. & Sejdinovic, D. Detecting and quantifying causal associations in large nonlinear time series datasets. Sci. Adv. 5, eaau4996 (2019).
Google Scholar
link