Accuracy of Four Language Analysis Procedures Performed Automatically Most software for language analysis has relied on an interaction between the metalinguistic skills of a human coder and the calculating ability of the machine to produce reliable results. However, probabilistic parsing algorithms are now capable of highly accurate and completely automatic identification of grammatical word classes. The program Computerized ... Research Article
Research Article  |   May 01, 2001
Accuracy of Four Language Analysis Procedures Performed Automatically
 
Author Affiliations & Notes
  • Steven H. Long
    Case Western Reserve University, Cleveland, OH
  • Ron W. Channell
    Brigham Young University, Provo, UT
  • Contact author: Steven H. Long, PhD, Department of Communication Sciences, Case Western Reserve University, 11206 Euclid Ave., Cleveland, OH 44106. E-mail: sxl12@po.cwru.edu
Article Information
Research Articles
Research Article   |   May 01, 2001
Accuracy of Four Language Analysis Procedures Performed Automatically
American Journal of Speech-Language Pathology, May 2001, Vol. 10, 180-188. doi:10.1044/1058-0360(2001/017)
History: Received August 17, 2000 , Accepted March 9, 2001
 
American Journal of Speech-Language Pathology, May 2001, Vol. 10, 180-188. doi:10.1044/1058-0360(2001/017)
History: Received August 17, 2000; Accepted March 9, 2001
Web of Science® Times Cited: 7

Most software for language analysis has relied on an interaction between the metalinguistic skills of a human coder and the calculating ability of the machine to produce reliable results. However, probabilistic parsing algorithms are now capable of highly accurate and completely automatic identification of grammatical word classes. The program Computerized Profiling combines a probabilistic parser with modules customized to produce four clinical grammatical analyses: MLU, LARSP, IPSyn, and DSS. The accuracy of these analyses was assessed on 69 language samples from typically developing, speech-impaired, and language-impaired children, 2 years 6 months to 7 years 10 months. Values obtained with human coding and by the software alone were compared. Results for all four analyses produced automatically were comparable to published data on the manual interrater reliability of these procedures. Clinical decisions based on cutoff scores and productivity data were little affected by the use of automatic rather than human-generated analyses. These findings bode well for future clinical and research use of automatic language analysis software.

Author Note
We thank Marc Fey, Lesley Olswang, and Sarah Woods for their contribution of the SLI, SELI, and Fluency language sample transcripts used in this study. Some of the findings of this study were reported at the Symposium on Research in Child Language Disorders, Madison, Wisconsin, 1997.
The software Computerized Profiling can be downloaded without charge from http://www.cwru.edu/artsci/cosi/cp.htm.
Order a Subscription
Pay Per View
Entire American Journal of Speech-Language Pathology content & archive
24-hour access
This Article
24-hour access