Using Vision and Large Language Models for Psychological Intervention Competency Assessment: A Pilot Study of Agreement with Clinician Ratings

Introduction: Supervision in psychological intervention training is limited by the time needed to evaluate simulated sessions. We tested a Vision Language Model (VLM) and Large Language Model (LLM) pipeline that assesses competency from verbal and nonverbal communication. Method: Gemini-3.1-Pro was used to annotate behaviors in 25 simulated therapy videos and incorporate them into transcripts, and ChatGPT-6.1-Sol rated them against the ENACT scale. Two clinicians verified VLM-identified behaviors and four independently rated each video. Agreement was estimated with Gwet's AC1 and ICC. Results:

This content is for informational purposes only and is not medical advice. It is not intended to diagnose, treat, cure, or prevent any disease. Consult a healthcare professional before starting any supplement, treatment, or program: especially if you are pregnant, nursing, taking medication, or managing a health condition.