Evaluation of an Artificial Intelligence-Based Medical Device Designed to Aid in the Diagnosis of Autism Spectrum Disorder in the Primary Care Setting

by Sharief Taraman

Medical Devices & Digital Health

Objectives: The lack of diagnostic tools for Autism Spectrum Disorder in primary care settings and long wait lists for specialist assessment contribute to an average delay of 3 years between first parental concern and diagnosis. This study examined the performance of an artificial intelligence-based device intended to aid PCPs in the diagnosis of ASD.

Methods: This was a prospective multi-site pivotal study conducted in 6 states using a double-blind active comparator design with 425 completed subjects (36% female) ages 18-72 months with concern for developmental delay. Previous research developed, tuned, and tested a device that uses a gradient boosted decision tree machine learning algorithm which analyzes 64 behavioral features from 3 distinct inputs: 1) Caregiver questionnaire 2) 2-4 minutes of home videos analyzed by trained video analysts 3) PCP questionnaire.

Device results were compared to diagnosis by independent agreement of specialist clinicians based on clinical assessment, including a modified CARS-2 and DSM-5 criteria. Specialists were child psychiatrists, child psychologists, pediatric neurologists, and developmental behavioral pediatricians experienced in diagnosing ASD.
Results Comparison of device results to specialist diagnosis found the PPV: 80.8% [95%CI, 70.3%-88.8%], NPV: 98.3% [90.6%-100%], sensitivity: 98.4% [91.6%-100%], specificity: 78.9% [67.6%-87.7%] for subjects with determinate device results. There was no evidence that device performance significantly varied when PCP used the device remotely compared to in-person.

Conclusions: Using this device, PCPs could efficiently, accurately, and equitably diagnose a subset of children 18-72 months old, thereby streamlining specialist referrals and facilitating earlier ASD diagnosis and interventions. The results further provide preliminary evidence that PCP evaluation of the child can be done via telemedicine or in-person with no degradation in device performance.