Automated Scoring of Nonnative Speech Using the  SpeechRater SM v. 5.0 Engine

[1]  Jian Cheng,et al.  Using deep neural networks to improve proficiency assessment for children English language learners , 2014, INTERSPEECH.

[2]  Kristin Precoda,et al.  EduSpeak®: A speech recognition and pronunciation scoring toolkit for computer-aided language learning applications , 2010 .

[3]  H. Riggenbach.  Toward an understanding of fluency: A microanalysis of nonnative speaker conversations , 1991 .

[4]  D. S. Boomer.  Hesitation and Grammatical Encoding , 1965, Language and speech.

[5]  Vassilios Digalakis,et al.  Automatic pronunciation evaluation of foreign speakers using unknown text , 2007, Comput. Speech Lang..

[6]  Derrick Higgins,et al.  EVALUATING THE CONSTRUCT‐COVERAGE OF THE E‐RATER® SCORING ENGINE , 2009 .

[7]  Klaus Zechner,et al.  Towards Automatic Scoring of Non-Native Spontaneous Speech , 2006, HLT-NAACL.

[8]  V. Dellwo,et al.  Comparing native and non-native speech rhythm using acoustic rhythmic measures: Cantonese, Beijing Mandarin and English , 2008 .

[9]  Wai Kit Lo,et al.  Statistical phone duration modeling to filter for intact utterances in a computer-assisted pronunciation training system , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[10]  Su-Youn Yoon,et al.  Vocabulary Profile as a Measure of Vocabulary Sophistication , 2012, BEA@NAACL-HLT.

[11]  Kristin Precoda,et al.  The SRI EduSpeak System: Recognition and Pronunciation Scoring for Language Learning , 2007 .

[12]  Jian Cheng,et al.  Fluency and structural complexity as predictors of L2 oral proficiency , 2010, INTERSPEECH.

[13]  Joel R. Tetreault,et al.  Using Entity-Based Features to Model Coherence in Student Essays , 2010, HLT-NAACL.

[14]  Xiaoming Xi,et al.  Improved pronunciation features for construct-driven assessment of non-native spontaneous speech , 2009, HLT-NAACL.

[15]  David M. Williamson,et al.  A Framework for Evaluation and Use of Automated Scoring , 2012 .

[16]  Klaus Zechner,et al.  Prompt-based Content Scoring for Automated Spoken Language Assessment , 2013, BEA@NAACL-HLT.

[17]  Peter W. Foltz,et al.  Textual coherence using latent semantic analysis , 1998 .

[18]  J. Goeman.  L1 Penalized Estimation in the Cox Proportional Hazards Model , 2009, Biometrical journal. Biometrische Zeitschrift.

[19]  Helen Yannakoudakis,et al.  Modeling coherence in ESOL learner texts , 2012, BEA@NAACL-HLT.

[20]  Xiaoming Xi,et al.  SpeechraterTM: a construct-driven approach to scoring spontaneous non-native speech , 2007, SLaTE.

[21]  Klaus Zechner,et al.  Exploring Content Features for Automated Speech Scoring , 2012, HLT-NAACL.

[22]  Joost van Doremalen,et al.  Utterance verification in language learning applications , 2009, SLaTE.

[23]  Lei Chen,et al.  Detecting Structural Events for Assessing Non-Native Speech , 2011, BEA@ACL.

[24]  E. Grabe,et al.  Durational variability in speech and the rhythm class hypothesis , 2005 .

[25]  Xiaoming Xi,et al.  Evaluating analytic scoring for the TOEFL® Academic Speaking Test (TAST) for operational use , 2007 .

[26]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[27]  P. Nation,et al.  Vocabulary size and use: Lexical richness in L2 written production , 1995 .

[28]  Helmer Strik,et al.  Modeling pronunciation variation for ASR: A survey of the literature , 1999, Speech Commun..

[29]  Anastassia Loukina,et al.  Feature selection for automated speech scoring , 2015, BEA@NAACL-HLT.

[30]  Mark J. F. Gales,et al.  Automatically grading learners' English using a Gaussian process , 2015, SLaTE.

[31]  Xiaoming Xi,et al.  A three-stage approach to the automated scoring of spontaneous spoken responses , 2011, Comput. Speech Lang..

[32]  Danqi Chen,et al.  of the Association for Computational Linguistics: , 2001 .

[33]  Noriko Iwashita,et al.  Features of oral proficiency in task performance by EFL and JFL learners , 2010 .

[34]  P. Holland,et al.  POPULATION INVARIANCE AND THE EQUATABILITY OF TESTS: BASIC THEORY AND THE LINEAR CASE , 2000 .

[35]  Mirella Lapata,et al.  Modeling Local Coherence: An Entity-Based Approach , 2005, ACL.

[36]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[37]  Anastassia Loukina,et al.  Rhythm measures and dimensions of durational variation in speech. , 2011, The Journal of the Acoustical Society of America.

[38]  Yong Wang,et al.  Improved mispronunciation detection with deep neural network trained acoustic models and transfer learning based logistic regression classifiers , 2015, Speech Commun..

[39]  Antony John Kunnan,et al.  The Duolingo English Test , 2015 .

[40]  Klaus Zechner,et al.  Applying rhythm metrics to non-native spontaneous speech , 2013, SLaTE.

[41]  F. Ramus,et al.  Correlates of linguistic rhythm in the speech signal , 1999, Cognition.

[42]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[43]  Frank K. Soong,et al.  A DNN-based acoustic modeling of tonal language and its application to Mandarin pronunciation training , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[44]  L. Boves,et al.  Quantitative assessment of second language learners' fluency by means of automatic speech recognition technology. , 2000, The Journal of the Acoustical Society of America.

[45]  Helmer Strik,et al.  Web-based mini-games for language learning that support spoken interaction , 2015, SLaTE.

[46]  P. Lennon.  Investigating Fluency in EFL: A Quantitative Approach* , 1990 .

[47]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[48]  Mitch Weintraub,et al.  Automatic scoring of pronunciation quality , 2000, Speech Commun..

[49]  Lei He,et al.  Syllabic intensity variations as quantification of speech rhythm: evidence from both L1 and L2 , 2012, Speech Prosody 2012.

[50]  Yang Liu,et al.  Coherence in child language narratives: a case study of annotation and automatic prediction of coherence , 2012, WOCCI.

[51]  Linda L. Cook,et al.  Fairness in Educational Assessment and Measurement , 2016 .

[52]  Mary P. Harper,et al.  Structural event detection for rich transcription of speech , 2004 .

[53]  Helmer Strik,et al.  My Pronunciation Coach: Computer-assisted English pronunciation training , 2014 .

[54]  Xiaoming Xi.  Evaluating analytic scoring for the TOEFL® Academic Speaking Test (TAST) for operational use , 2007 .

[55]  L. Boves,et al.  Quantitative assessment of second language learners' fluency: comparisons between read and spontaneous speech. , 2002, The Journal of the Acoustical Society of America.

[56]  Laurence White,et al.  Calibrating rhythm: First language and second language studies , 2007, J. Phonetics.

[57]  Sid-Ahmed Selouani,et al.  Native and non-native class discrimination using speech rhythm- and auditory-based cues , 2015, Comput. Speech Lang..

[58]  Su-Youn Yoon,et al.  Automatic assessment of syntactic complexity for spontaneous speech scoring , 2015, Speech Commun..

[59]  Ani Nenkova,et al.  Automatic Evaluation of Linguistic Quality in Multi-Document Summarization , 2010, ACL.

[60]  W. Bruce Croft,et al.  Similarity measures for tracking information flow , 2005, CIKM '05.

[61]  Regina Barzilay,et al.  Catching the Drift: Probabilistic Content Models, with Applications to Generation and Summarization , 2004, NAACL.

[62]  Marc A. Zissman,et al.  Comparison of four approaches to automatic language identification of telephone speech , 1996, IEEE Trans. Speech Audio Process..

[63]  Xiaoming Xi,et al.  Towards Automatic Scoring of a Test of Spoken Language with Heterogeneous Task Types , 2008 .

[64]  Mirella Lapata,et al.  Modeling Local Coherence: An Entity-Based Approach , 2005, ACL.

[65]  Su-Youn Yoon,et al.  Application of Structural Events Detected on ASR Outputs for Automated Speaking Assessment , 2012, INTERSPEECH.

[66]  Lei Chen,et al.  Applying Rhythm Features to Automatically Assess Non-Native Speech , 2011, INTERSPEECH.

[67]  Su-Youn Yoon,et al.  Non-English Response Detection Method for Automated Proficiency Scoring System , 2011, BEA@ACL.

[68]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.

[69]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[70]  David M. Williamson,et al.  Automated essay scoring: Psychometric guidelines and practices , 2013 .

[71]  Daniel Marcu,et al.  Evaluating Multiple Aspects of Coherence in Student Essays , 2004, NAACL.

[72]  Ronald A. Cole,et al.  The OGI multi-language telephone speech corpus , 1992, ICSLP.

[73]  Anastassia Loukina,et al.  Automatic evaluation of spoken summaries: the case of language assessment , 2014, BEA@ACL.

[74]  Liz Temple,et al.  Second language learner speech production , 2000 .

[75]  R. Hout,et al.  Lexical richness in the spontaneous speech of bilinguals , 2003 .

[76]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[77]  Helmer Strik,et al.  Comparing different approaches for automatic pronunciation error detection , 2009, Speech Commun..

[78]  M. Sanderson,et al.  Duplicate Detection in the Reuters Collection , 1997 .

[79]  Xin Chen,et al.  Deep neural network acoustic models for spoken assessment applications , 2015, Speech Commun..

[80]  Laurence White,et al.  Language categorization by adults is based on sensitivity to durational cues, not rhythm class , 2012 .

[81]  Xiaofei Lu,et al.  Automatic analysis of syntactic complexity in second language writing , 2010 .

[82]  Keelan Evanini,et al.  The influence of automatic speech recognition accuracy on the performance of an automated speech assessment system , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[83]  Jian Cheng,et al.  Off-Topic Detection in Automated Speech Assessment Applications , 2011, INTERSPEECH.

[84]  A. Vermeer.  Coming to grips with lexical richness in spontaneous speech data , 2000 .

[85]  Kathryn Bock,et al.  Language production : Grammatical encoding , 1994 .

[86]  Shunji Inagaki,et al.  Second Language Development in Writing: Measures of Fluency, Accuracy, and Complexity , 1998 .

[87]  Peter W. Foltz,et al.  The Measurement of Textual Coherence with Latent Semantic Analysis. , 1998 .

[88]  Larry S. Davis,et al.  Automated Scoring of Speaking Tasks in the Test of English‐for‐Teaching (TEFT™) , 2015 .

[89]  Beata Beigman Klebanov,et al.  Content Importance Models for Scoring Writing From Sources , 2014, ACL.

[90]  Su-Youn Yoon,et al.  Assessment of ESL Learners' Syntactic Competence Based on Similarity Measures , 2012, EMNLP-CoNLL.

[91]  Arthur C. Graesser,et al.  Coh-Metrix: Analysis of text on cohesion and language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[92]  Maxine Eskénazi,et al.  An overview of spoken language technology for education , 2009, Speech Commun..

[93]  Shrikanth S. Narayanan,et al.  Connecting rhythm and prominence in automatic ESL pronunciation scoring , 2009, INTERSPEECH.

[94]  L. Ortega.  Syntactic Complexity Measures and Their Relationship to L2 Proficiency: A Research Synthesis of College-Level L2 Writing. , 2003 .

[95]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[96]  Klaus Zechner,et al.  What did they actually say? agreement and disagreement among transcribers of non-native spontaneous speech responses in an English proficiency test , 2009, SLaTE.

[97]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[98]  Klaus Zechner,et al.  Computing and Evaluating Syntactic Complexity Features for Automated Scoring of Spontaneous Non-Native Speech , 2011, ACL.

[99]  Xiaoming Xi,et al.  Automated Scoring of Spontaneous Speech Using SpeechRater? v1.0. Research Report. ETS RR-08-62. , 2008 .

[100]  Jian Cheng,et al.  Validating automated speaking tests , 2010 .

[101]  Justin Zobel,et al.  Methods for Identifying Versioned and Plagiarized Documents , 2003, J. Assoc. Inf. Sci. Technol..

[102]  Su-Youn Yoon,et al.  Automated Scoring of Nonnative Speech Using the "SpeechRater"? v. 5.0 Engine. Research Report. ETS RR-18-10. , 2018 .

[103]  Klaus Zechner,et al.  Coherence Modeling for the Automated Assessment of Spontaneous Spoken Responses , 2013, NAACL.

[104]  Helmer Strik,et al.  Automatic evaluation of Dutch pronunciation by using speech recognition technology , 1997, 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings.

[105]  T. McNamara,et al.  Assessed Levels of Second Language Speaking Proficiency: How Distinct? , 2007 .

[106]  Xiaoming Xi,et al.  Automatic scoring of non-native spontaneous speech in tests of spoken English , 2009, Speech Commun..

[107]  Elmar Nöth,et al.  Automatic assessment of non-native prosody , 2009, SLaTE.

[108]  Elmar Nöth,et al.  Pronunciation Feature Extraction , 2005, DAGM-Symposium.

[109]  Frank K. Soong,et al.  The Use of DBN-HMMs for Mispronunciation Detection and Diagnosis in L2 English to Support Computer-Aided Pronunciation Training , 2012, INTERSPEECH.