Creative writing is orders of magnitude easier to detect. The biggest problem for AI writing detection tools is the use of translation — so generating text, translating it into another language, and translating it back — but this ends up as a false-negative issue. False positives are very rare.
More importantly, we don’t need GPTZero to detect AI writing because professionals can identify it on sight. Add context clues (students are dumb) and suddenly it’s trivial.
EDIT: I need you to understand that college students are terrible writers. In all my decades and literally thousands of students, I’ve met maybe three undergrads whose writing was more than proficient, and I work with students whose SAT scores are over 1550. The idea I’m going to mistake your writing for AI (which sounds horrible and robotic, btw) because of its “high quality” is simply unfathomable.
The false positive rate is pretty low, but a false negative of 35% is pretty concerning in terms of whether they’re fit for purpose. Like it’ll rarely misclassify human writing as AI generated, but it’s somewhat closer to a coin flip in correctly identifying AI generated language. It’s a signal, but I imagine it’s a poor substitute for the judgement of a person who has spent time teaching teenagers / young adults to write.
Which to the original point isn’t totally snake oil, but it’s maybe not a great tool for the professionals who it’s targeting.
The false negative rate for creative writing is exceedingly low as well, sub 1%. False negatives aren’t the critique of this meme, and the examples I gave for GPTZero were worst case scenarios for very specific types of technical writing.
Worst performance for GPTZero is technical medical papers with a 10% false positive rate. Here’s the study from S. Korea.
Scientific scholarly writing: <4% false positive.
Creative writing is orders of magnitude easier to detect. The biggest problem for AI writing detection tools is the use of translation — so generating text, translating it into another language, and translating it back — but this ends up as a false-negative issue. False positives are very rare.
More importantly, we don’t need GPTZero to detect AI writing because professionals can identify it on sight. Add context clues (students are dumb) and suddenly it’s trivial.
EDIT: I need you to understand that college students are terrible writers. In all my decades and literally thousands of students, I’ve met maybe three undergrads whose writing was more than proficient, and I work with students whose SAT scores are over 1550. The idea I’m going to mistake your writing for AI (which sounds horrible and robotic, btw) because of its “high quality” is simply unfathomable.
Props for responding constructively.
The false positive rate is pretty low, but a false negative of 35% is pretty concerning in terms of whether they’re fit for purpose. Like it’ll rarely misclassify human writing as AI generated, but it’s somewhat closer to a coin flip in correctly identifying AI generated language. It’s a signal, but I imagine it’s a poor substitute for the judgement of a person who has spent time teaching teenagers / young adults to write.
Which to the original point isn’t totally snake oil, but it’s maybe not a great tool for the professionals who it’s targeting.
The false negative rate for creative writing is exceedingly low as well, sub 1%. False negatives aren’t the critique of this meme, and the examples I gave for GPTZero were worst case scenarios for very specific types of technical writing.