Andrew Searles (University of California Irvine), Renascence Tarafder Prapty (University of California Irvine), Gene Tsudik (University of California Irvine)

Since 2003, CAPTCHAS have been widely used as a barrier against bots, while simultaneously annoying great multitudes of users worldwide. As the use of CAPTCHAS grew, techniques to defeat or bypass them kept improving. In response, CAPTCHAS themselves evolved in terms of sophistication and diversity, becoming increasingly difficult to solve for both bots and humans. Given this long-standing and still-ongoing arms race, it is important to investigate usability, solving performance, and user perceptions of modern CAPTCHAS. In this work, we do so via a large scale (over 3,600 distinct users) 13-month realworld user study and post-study survey. The study, conducted at a large public university, is based on a live account creation and password recovery service with currently prevalent CAPTCHA type: reCAPTCHAv2.

Results show that, with more attempts, users improve in solving checkbox CAPTCHAS. For website developers and user study designers, results indicate that the website context, i.e., whether the service is password recovery or account creation, directly influences (with statistically significant differences) CAPTCHA solving times. We consider the impact of participants’ major and education level, showing that certain majors exhibit better performance, while, in general, education level has a direct impact on solving time. Unsurprisingly, we discover that participants find image CAPTCHAS to be annoying, while checkbox CAPTCHAS are perceived as easy. We also show that, rated via System Usability Scale (SUS), image CAPTCHAS are viewed as “OK”, while checkbox CAPTCHAS are viewed as “good”.

Finally, we also explore the cost and security of reCAPTCHAv2 and conclude that it comes at an immense cost and offers practically no security. Overall, we believe that this study’s results prompt a natural conclusion: reCAPTCHAv2 and similar reCAPTCHA technology should be deprecated.

View More Papers

“So I Sold My Soul“: Effects of Dark Patterns...

Oksana Kulyk (ITU Copenhagen), Willard Rafnsson (IT University of Copenhagen), Ida Marie Borberg, Rene Hougard Pedersen

Read More

IoT Software Updates: User Perspectives in the Context of...

S. P. Veed, S. M. Daftary, B. Singh, M. Rudra, S. Berhe (University of the Pacific), M. Maynard (Data Independence LLC) F. Khomh (Polytechnique Montreal)

Read More

Understanding Influences on SMS Phishing Detection: User Behavior, Demographics,...

Daniel Timko (California State University San Marcos), Daniel Hernandez Castillo (California State University San Marcos), Muhammad Lutfor Rahman (California State University San Marcos)

Read More

PropertyGPT: LLM-driven Formal Verification of Smart Contracts through Retrieval-Augmented...

Ye Liu (Singapore Management University), Yue Xue (MetaTrust Labs), Daoyuan Wu (The Hong Kong University of Science and Technology), Yuqiang Sun (Nanyang Technological University), Yi Li (Nanyang Technological University), Miaolei Shi (MetaTrust Labs), Yang Liu (Nanyang Technological University)

Read More