Testing Public Speaking Proficiency: Designing an Assessment Instrument for University Students
DOI:
https://doi.org/10.32528/issh.v2i2.261Keywords:
assessment instrument, proficiency test, public speaking, speaking proficiency, R&D designAbstract
This study aimed to develop a valid and reliable assessment instrument to measure university students' public speaking proficiency. A sample of 30 students from a private university in East Java, Indonesia, was selected based on inclusion and exclusion criteria. The assessment instrument consisted of five main components, and the reliability and validity of the instrument were assessed through various statistical methods. Results showed that the instrument effectively measured public speaking proficiency, with students performing well on fluency and eye contact and gestures. However, students struggled the most with the organization and structure component. The findings suggest that the assessment instrument can provide useful feedback for improvement in specific areas and offers a valuable tool for evaluating university students' public speaking proficiency. This information can be critical for enhancing teaching practices. Further research is recommended to validate its use in different settings and populations.
Downloads
References
Ahmad, H., & Halim, H. (2017). Determining sample size for research activities. Selangor Business Review, 20–34.
Ajani, K. M. (2021). They Who Hesitate: The Influencing Factors of Hesitation Phenomena in Students’ Speaking at IAIN Langsa. JADEs Journal of Academia in English Education, 2(2), 110–134.
Altun, M. (2019). An underestimated tool: Body language in classroom during teaching and learning. International Journal of Social Sciences & Educational Studies, 6(1), 155-170.
Amidei, J., Piwek, P., & Willis, A. (2019). The use of rating and Likert scales in Natural Language Generation human evaluation tasks: A review and some recommendations.
Bauth, M. F., Angélico, A. P., & Oliveira, D. C. R. de. (2019). Association between social skills, sociodemographic factors and self-statements during public speaking by university students. Trends in Psychology, 27, 677–692.
Blaikie, N. (2018). Confounding issues related to determining sample size in qualitative research. International Journal of Social Research Methodology, 21(5), 635–641.
Boddy, C. R. (2016). Sample size for qualitative research. Qualitative Market Research: An International Journal, 19(4), 426–432.
Bonvillain, N. (2019). Language, culture, and communication: The meaning of messages. Rowman & Littlefield.
Bradberry, L. A., & de Maio, J. (2019). Learning by doing: The long-term impact of experiential learning programs on student success. Journal of Political Science Education, 15(1), 94–111.
Braithwaite, D. O., Schrodt, P., & Phillips, K. E. (2021). Introduction: Meta-theory and theory in interpersonal communication research. In Engaging theories in interpersonal communication (pp. 1–23). Routledge.
Burns, A. (2016). Teaching speaking: Towards a holistic approach. 25th ETA-ROC Anniversary Conference: Epoch Making in English Language Teaching and Learning. Taiwan: Chien Tan Overseas Youth Activity Center, November.
Burns, A. (2019). Concepts for teaching speaking in the English language classroom. LEARN Journal: Language Education and Acquisition Research Network, 12(1), 1–11.
de Oliveira, A. R., & Proença, A. (2019). Design principles for research & development performance measurement systems: a systematic literature review. Brazilian Journal of Operations & Production Management, 16(2), 227–240.
Din, K. U., & Saeed, M. (2018). Relationship between University Students’ English Proficiency, academic achievement and their satisfaction on teacher feedback. Bulletin of Education and Research, 40(3), 129–143.
Dinh, T. B. N., & Tran, T. D. (2020). KEY FACTORS INFLUENCING LEARNERS’ORAL FLUENCY IN ENGLISH SPEAKING CLASSES: A CASE AT A PUBLIC UNIVERSITY IN VIET NAM. VNU Journal of Foreign Studies, 36(6).
Dondi, M., Klier, J., Panier, F., & Schubert, J. (2021). Defining the skills citizens will need in the future world of work. McKinsey & Company, 25.
Fan, J., & Yan, X. (2020). Assessing speaking proficiency: a narrative review of speaking assessment research within the argument-based validation framework. Frontiers in Psychology, 11, 330.
Ferrer-Pardo, V. R., Jimenez-Perez, I., Gil-Calvo, M., Pérez-Soriano, P., & Priego-Quesada, J. I. (2022). Relationship between Students’ Perception of a Rubric for Oral Presentations and Their Academic Characteristics. Education Sciences, 12(11), 765.
Gallego, A., McHugh, L., Villatte, M., & Lappalainen, R. (2020). Examining the relationship between public speaking anxiety, distress tolerance and psychological flexibility. Journal of Contextual Behavioral Science, 16, 128–133.
Gan, Z., Hu, G., Wang, W., Nang, H., & An, Z. (2021). Feedback behaviour and preference in university academic English courses: associations with English language self-efficacy. Assessment & Evaluation in Higher Education, 46(5), 740–755.
Gani, S. A., Fajrina, D., & Hanifa, R. (2015). Students’ learning strategies for developing speaking ability. Studies in English Language and Education, 2(1), 16–28.
Hall, S. (1997). Integrating Pronunciation for Fluency in Presentation Skills.
Horii, K., Fukuda, M., Ohta, K., Nishimura, R., Ogawa, A., & Kitaoka, N. (2021). End-to-End Spontaneous Speech Recognition Using Hesitation Labeling. 2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 1077–1081.
John, A. D., Nagarajan, G., & Arthi, M. (2017). Non Verbal Communication in Public Speaking. Impact Journals, 5(2), 97-100.
Kelsen, B. A. (2019). Exploring public speaking anxiety and personal disposition in EFL presentations. Learning and Individual Differences, 73, 92–101.
Khannan, M. S. A., Tontowi, A. E., Herliansyah, M. K., & Sudiarso, A. (2021). New Product Development Method Trends and Future Research: A Systematic Literature Review. Jurnal Teknik Industri, 23(1), 11–24.
Kusi-Mensah, K., Nuamah, N. D., Wemakor, S., Agorinya, J., Seidu, R., Martyn-Dickens, C., & Bateman, A. (2022). A systematic review of the validity and reliability of assessment tools for executive function and adaptive function following brain pathology among children and adolescents in low-and middle-income countries. Neuropsychology Review, 32(4), 974–1016.
Leong, L. M., & Ahmadi, S. M. (2017). An analysis of factors influencing learners’ English speaking skill.
Levis, J. M. (2018). Intelligibility, oral communication, and the teaching of pronunciation. Cambridge University Press.
Ma, R. (2015). The role of pronunciation in speaking test ratings. Brigham Young University.
Ma, R., Henrichsen, L. E., Cox, T. L., & Tanner, M. W. (2018). Pronunciation’s role in English speaking-proficiency ratings. Journal of Second Language Pronunciation, 4(1), 73–102.
Malterud, K., Siersma, V., & Guassora, A. D. (2021). Information power: Sample content and size in qualitative studies.
Maryam, I. S., Febriani, R. B., & Kurnia, A. D. (2019). Efl learners’ perceptions towards their self-efficacy in learning public speaking course. Journal of English Education and Teaching, 3(3), 377–391.
Naqvi, S., Srivastava, R., al Damen, T., al Aufi, A., al Amri, A., & al Adawi, S. (2023). Establishing Reliability and Validity of an Online Placement Test in an Omani Higher Education Institution. Languages, 8(1), 61.
Pitt, E., Bearman, M., & Esterhazy, R. (2019). The conundrum of low achievement and feedback for learning. Assessment & Evaluation in Higher Education.
Qadhi, A. S. (2018). Instructional strategies to develop the speaking skill. International Journal of Linguistics, Literature and Translation, 1(3), 33–36.
Ramos, I. D. (2020). Public speaking preparation stage: Critical thinking and organization skills in South Korea. International Research in Education, 8(2), 77–96.
Rao, P. S. (2019). The importance of speaking skills in English classrooms. Alford Council of International English & Literature Journal (ACIELJ), 2(2), 6–18.
Saito, K., & Plonsky, L. (2019). Effects of second language pronunciation teaching revisited: A proposed measurement framework and meta‐analysis. Language Learning, 69(3), 652–708.
Stephenson, M., & Hall, G. (2021). Organizing talk in group speaking tests: learning from high-scoring students. ELT Journal, 75(1), 42–54.
Susanto, A., Halim, F. A. B., & Thasimmim, S. N. (2019). Vocabulary learning strategies, vocabulary skills, and integrative motivation levels among university students. Vocabulary Learning Strategies, Vocabulary Skills, and Integrative Motivation Levels among University Students, 8(5C), 323–334.
Suzuki, S., Kormos, J., & Uchihara, T. (2021). The relationship between utterance and perceived fluency: A meta‐analysis of correlational studies. The Modern Language Journal, 105(2), 435–463.
Tavakoli, P., Nakatsuhara, F., & Hunter, A. (2020). Aspects of fluency across assessed levels of speaking proficiency. The Modern Language Journal, 104(1), 169–191.
Tekşan, K., Mutlu, H. H., & Çinpolat, E. (2019). The examination of the relationship between the speech anxiety and speaking skill attitudes of middle school students and the opinions of teachers on speech anxiety. Journal of Language and Linguistic Studies, 15(4), 1395–1412.
Underhill, J. C., Ledford, V., & Adams, H. M. (2021). ‘Public Speaking is a Skill that Everyone Needs No Matter What’: Exploring Peer Perceptions toward Students on the Autism Spectrum in Basic Course Classrooms. Basic Communication Course Annual, 33(1), 8.
Zainurrahman, Z., & Sangaji, S. (2019). A study on the university students’ speaking difficulties. Langua: Journal of Linguistics, Literature, and Language Education, 2(1), 1–8.
Zhang, L., & Lo, Y. Y. (2021). EMI teachers’ use of interactive metadiscourse in lecture organisation and knowledge construction. In Language use in English-medium instruction at university (pp. 56-79). Routledge.
Zhang, X., & Ardasheva, Y. (2019). Sources of college EFL learners’ self-efficacy in the English public speaking domain. English for Specific Purposes, 53, 47–59.
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Indah Werdiningsih, Nur Mukminatien
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
