Investigating Semantic Roles in Visual Search

In our daily lives, we frequently engage in effective search within naturalistic scenes, implying that object and scene recognition often require minimal attentional resources. Disrupting scene regularities (e.g., placing a fire hydrant in a kitchen) is a common method to illustrate how scene context influences processes like object recognition, visual search, memorization, and action.

Drawing inspiration from the SCEGRAM image dataset (Öhlschläger & Võ, 2017), this project included the building of a dataset consisting of semantically congruent and incongruent objects with the scene context.

Previous research has extensively studied how behavior is shaped by the properties of our goals and targets. Examining the factors influencing attention and action in real-world contexts is crucial for understanding natural human behaviors in relation to the visual external world. Against this backdrop, the combination of an incongruent target paired with a congruent distractor has not thoroughly been studied, presenting an exciting opportunity to further investigate how the semantic roles of scenes and objects impact search, as demonstrated in Figure 1D.

Figure 1: Indoor table scene from the image dataset. A: congruent target (bread), congruent distractor (oranges). B: congruent target (bread), incongruent distractor (golf balls). C: incongruent target (wallet), incongruent distractor (golf balls). D: incongruent target (wallet), congruent distractor (oranges).

A broad video overview of this project, created for the UC Davis Research Rockstars Undergrad Slam Contest!