We gave AI detectors a try–here’s what we found

Key points:

AI detection tools are skyrocketing in popularity–but how efficient are they?

A look at different AI detectors offers an eye-opening look at whether or not AI-generated pieces are identified as such

See related article: Is AI the future of education?

Nearly every school or university faculty is having at least a few conversations about how to address a world rich in easy-to-use artificial intelligence tools that can generate student assignments.

Multiple AI detection services claim efficacy in identifying whether text is generated by AI or human writers. Turnitin, ZeroGPT, Quill, and AI Textclassifier each represent this ability and are in use by higher-ed faculty and K-12 educators.

In an attempt to determine the effectiveness of Turnitin’s ability to identify artificial intelligence generated materials, students in a doctoral methods course were asked to submit one or two assignments that were fully generated by ChatGPT or another generative tool like Google’s Bard or Microsoft’s Bing AI. It appears that most students used ChatGPT. Of 28 fully AI-derived assignments, 24 of 28 were determined to be 100 percent AI generated. The other four ranged from zero to 65 percent AI-derived. The size of the papers ranged from 411 to 1368 words.

Turnitin returned evidence of potential plagiarism through its Similarity Scores in the range from zero percent to 49 percent. The average AI generated paper was noted to be 13.75 percent similar to other extant materials. (You can find Turnitin’s AI Writing detection tool FAQ here.)

Matthew Flugum, Doctoral Student of Education at Winona State University and Digital Learning Coordinator, Edina Public Schools & Steven M. Baule, Ed.D., Ph.D., Faculty Member, Winona State University

Matthew Flugum is a doctoral student of Education through Winona State University and the Digital Learning Coordinator for Edina Public Schools in Minnesota focused on supporting educators to uncover future ready opportunities for student learning. This focus extends through leadership in curriculum design, digital learning integration, professional development, and adjunct supervision by helping educators transition to the new realities impacting student learning.
Dr. Steve Baule is a faculty member at Winona State University (WSU), where he teaches in the Leadership Education Department. Prior to joining WSU, Baule spent 28 years in K-12 school systems in Illinois, Indiana, and Iowa, and two years teaching in the University of Wisconsin System. For the 13 years prior to moving to the university level, Baule served as a public -school superintendent. He has written 10 books on a variety of educational and historical topics and has served on the editorial boards for two journals. Baule earned an advanced diversity and equity certificate while in the UW system. He holds a doctorate in instructional technology from Northern Illinois University and a doctorate in educational leadership and policy studies from Loyola University Chicago.

Latest posts by eSchool Media Contributors (see all)

Source: https://www.eschoolnews.com/digital-learning/2023/07/03/we-gave-ai-detectors-a-try-heres-what-we-found/

We gave AI detectors a try–here’s what we found

Welcome to

Accessibility Dashboard