These three experiments examined noticing rates of an unexpected object (UO) that appeared during a dynamic aircraft threat evaluation task that required participants to shift their visual attention between multiple task-relevant locations. Experiment 1 manipulated the location at which the UO appeared; no effects on noticing rates were found. However, eye-tracking data revealed trends for UOs to be noticed more when participants were looking at locations closer to where the UO appeared, or when they were making more eye-movements while the UO was present. Eye-tracking data also showed a strong link between making an eye movement to the UO and noticing it. Experiment 2 manipulated the color, direction and speed of the UO to make it more or less similar to task-relevant objects. Also, to-be-ignored (TBI) aircraft were either present or absent for each participant. An interaction between the color of the UO and the presence of TBI aircraft was found with noticing rates being greater for uniquely-colored UO's only when no TBI aircraft were present. No overall effect of UO and target aircraft similarity was found. Experiment 3 manipulated the visual complexity and cognitive difficulty of the task. Noticing rates were higher only in the visually-simple, cognitively-easy, task pairing. These findings reveal the importance of participants' task strategies, attentional set and the interaction with task complexity unexplored by current theories of visual attention and prior findings from research on inattentional blindness. Also discussed are the implications for designers of human-machine systems.