Improving Human-Machine Cooperative Visual Search With Soft Highlighting

Advances in machine learning have produced systems that attain human-level performance on certain visual tasks, e.g., object identification. Nonetheless, other tasks requiring visual expertise are unlikely to be entrusted to machines for some time, e.g., satellite and medical imagery analysis. We describe a human-machine cooperative approach to visual search, the aim of which is to outperform either human or machine acting alone. The traditional route to augmenting human performance with automatic classifiers is to draw boxes around regions of an image deemed likely to contain a target. Human experts typically reject this type of hard highlighting. We propose instead a soft highlighting technique in which the saliency of regions of the visual field is modulated in a graded fashion based on classifier confidence level. We report on experiments with both synthetic and natural images showing that soft highlighting achieves a performance synergy surpassing that attained by hard highlighting.

[1]  Trafton Drew,et al.  When and why might a computer-aided detection (CAD) system interfere with visual search? An eye-tracking study. , 2012, Academic radiology.

[2]  C. Lehman,et al.  Diagnostic Accuracy of Digital Screening Mammography With and Without Computer-Aided Detection. , 2015, JAMA internal medicine.

[3]  J. L. van Genderen,et al.  Remote Sensing: Statistical Testing of Thematic Map Accuracy , 1978 .

[4]  Lucy Bastin,et al.  Visualizing uncertainty in multi-spectral remotely sensed imagery , 2002 .

[5]  Michael C. Mozer,et al.  Perception of multiple objects - a connectionist approach , 1991, Neural network modeling and connectionism.

[6]  J R Anderson LAND - USE CLASSIFICATION SCHEMES , 1971 .

[7]  Chein-I Chang,et al.  A computer-aided design mammography screening system for detection and classification of microcalcifications , 2000, Int. J. Medical Informatics.

[8]  M. Giger,et al.  An "intelligent" workstation for computer-aided diagnosis. , 1993, Radiographics : a review publication of the Radiological Society of North America, Inc.

[9]  John A. Richards,et al.  Remote Sensing Digital Image Analysis: An Introduction , 1999 .

[10]  Lubomir M. Hadjiiski,et al.  Effect of CAD on radiologists' detection of lung nodules on thoracic CT scans: analysis of an observer performance study by nodule size. , 2009, Academic radiology.

[11]  C J D'Orsi,et al.  Computer-aided detection: there is no free lunch. , 2001, Radiology.

[12]  Nico Karssemeijer,et al.  Computer-aided detection of masses at mammography: interactive decision support versus prompts. , 2013, Radiology.

[13]  M. Masson Using confidence intervals for graphically based data interpretation. , 2003, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[14]  Regina A. Pomranky,et al.  The role of trust in automation reliance , 2003, Int. J. Hum. Comput. Stud..

[15]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[16]  David Gur,et al.  Detection and classification performance levels of mammographic masses under different computer-aided detection cueing environments1☆ , 2004 .

[17]  Mark Gahegan,et al.  Visualizing Geospatial Information Uncertainty: What We Know and What We Need to Know , 2005 .

[18]  Manpreet Kaur,et al.  Survey of Contrast Enhancement Techniques based on Histogram Equalization , 2011 .

[19]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[20]  L. Philpotts,et al.  Can computer-aided detection be detrimental to mammographic interpretation? , 2009, Radiology.

[21]  Antje Baer,et al.  Handbook Of Medical Image Processing And Analysis , 2016 .

[22]  T. Freer,et al.  Screening mammography with computer-aided detection: prospective study of 12,860 patients in a community breast center. , 2001, Radiology.

[23]  N. Karssemeijer,et al.  Analysis of probed regions in an interactive CAD system for the detection of masses in mammograms , 2009, Medical Imaging.

[24]  E. Krupinski,et al.  Perceptual enhancement of tumor targets in chest X-ray images , 1993, Perception & psychophysics.

[25]  Berkman Sahiner,et al.  Computerized detection and classification of microcalcifications on mammograms , 1995, Medical Imaging.

[26]  T KneuselRonald,et al.  Improving Human-Machine Cooperative Visual Search With Soft Highlighting , 2017 .

[27]  Heng-Da Cheng,et al.  Computer-aided detection and classification of microcalcifications in mammograms: a survey , 2003, Pattern Recognit..

[28]  B. Zheng,et al.  Soft-copy mammographic readings with different computer-assisted detection cuing environments: preliminary findings. , 2001, Radiology.

[29]  Lorenzo Strigini,et al.  Effects of incorrect computer-aided detection (CAD) output on human decision-making in mammography. , 2004, Academic radiology.

[30]  Nico Karssemeijer,et al.  Using computer-aided detection in mammography as a decision support , 2010, European Radiology.

[31]  David Gur,et al.  Detection and classification performance levels of mammographic masses under different computer-aided detection cueing environments. , 2004, Academic radiology.

[32]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[33]  Ali Nodehi,et al.  Using Triangular Function To Improve Size Of Population In Quantum Evolution Algorithm For Fractal Image Compression , 2011 .

[34]  K. Kinkel,et al.  Computer-aided detection (CAD) in mammography: does it help the junior or the senior radiologist? , 2005, European journal of radiology.

[35]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[36]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[37]  Hengl Tomislav,et al.  Maps are Not What They Seem: Representing Uncertainty in Soil-property Maps , 2006 .

[38]  Linda C. van der Gaag,et al.  Visual exploration of uncertainty in remote-sensing classification , 1998 .

[39]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[40]  Ronald T. Kneusel,et al.  Improving Human-Classifier Interaction through Enhanced Highlighting Techniques , 2016 .

[41]  Trafton Drew,et al.  Analog Computer-Aided Detection (CAD) information can be more effective than binary marks , 2016, Attention, Perception, & Psychophysics.

[42]  M. Raza,et al.  Brain Image Enhancement - A Survey , 2012 .