Building Segmentation Based Human-Friendly Human Interaction Proofs (HIPs)

Human interaction proofs (HIPs) have become common place on the internet due to their effectiveness in deterring automated abuse of online services intended for humans. However, there is a co-evolutionary arms race in progress and these proofs are becoming more difficult for genuine users while attackers are getting better at breaking existing HIPs. We studied various popular HIPs on the internet to understand their strength and human friendliness. To determine HIP strength, we adopted a direct approach of building computer attacks using image processing and machine learning techniques. To understand human-friendliness, a sequence of users studies were conducted to investigate HIP character recognition by humans under a variety of visual distortions and clutter commonly employed in reading-based HIPs. We found that many of the online HIPs are pure recognition tasks that can be easily broken using machine learning. The stronger HIPs tend to pose a combination of segmentation and recognition challenges. Further, the HIP user studies show that given correct segmentation, computers are much better at HIP character recognition than humans. In light of these results, we propose that segmentation-based reading challenges are the future for building stronger human-friendly HIPs. An example of such a segmentation-based HIP is presented with a preliminary assessment of its strength and human-friendliness.

[1]  Björn Stenger,et al.  Shape context and chamfer matching in cluttered scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[2]  O. G. Selfridge,et al.  Pandemonium: a paradigm for learning , 1988 .

[3]  Mary Czerwinski,et al.  Designing human friendly human interaction proofs (HIPs) , 2005, CHI.

[4]  Rachid Deriche,et al.  Fast algorithms for low-level vision , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[5]  Joshua Goodman,et al.  Stopping outgoing spam , 2004, EC '04.

[6]  Gabriel Moy,et al.  Distortion estimation techniques in solving visual CAPTCHAs , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[7]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[8]  Henry S. Baird.  Protecting Websites with Reading-Based CAPTCHAs , 2003 .

[9]  Jitendra Malik,et al.  Recognizing objects in adversarial clutter: breaking a visual CAPTCHA , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[10]  Patrice Y. Simard,et al.  Using Machine Learning to Break Visual Human Interaction Proofs (HIPs) , 2004, NIPS.

[11]  Henry S. Baird,et al.  BaffleText: a Human Interactive Proof , 2003, IS&T/SPIE Electronic Imaging.

[12]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[13]  K. S. Baird,et al.  Anatomy of a versatile page reader , 1992, Proc. IEEE.

[14]  Henry S. Baird,et al.  Pessimal print: a reverse Turing test , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[15]  Kris Popat,et al.  Human Interactive Proofs and Document Image Analysis , 2002, Document Analysis Systems.