Factors Affecting Audiovisual Speech Perception as Measured by the McGurk Effect

Simple item page
dc.contributor.advisorBeauchamp, Michael Sen_US
dc.contributor.advisorDannemiller, James Len_US
dc.creatorBasu Mallick, Debshilaen_US
dc.date.accessioned2017-08-02T19:03:32Zen_US
dc.date.available2017-08-02T19:03:32Zen_US
dc.date.created2016-05en_US
dc.date.issued2016-04-13en_US
dc.date.submittedMay 2016en_US
dc.date.updated2017-08-02T19:03:32Zen_US
dc.description.abstractMultisensory speech perception occurs when an individual integrates spoken sounds and mouth movements of a talker into a coherent percept, e.g., during face-to-face conversations. Under usual circumstances, spoken sounds and mouth movements match. However, when there is a mismatch between spoken sounds and mouth movements, individuals sometimes perceive a “fused” percept, different from the constituent audiovisual information. This phenomenon, known as the McGurk effect has been used in thousands of papers in the literature as a measure of audiovisual integration in speech. For my dissertation I attempted to extend the findings of my previous work by investigating the sources of interindividual and interstimulus differences in the McGurk effect. In the first experiment, I attempted to investigate the influence of response-type on individuals’ perception of the McGurk effect. Studies of the McGurk effect have predominantly adopted either an open-choice or a forced-choice response format to record participants’ responses. For my dissertation, I compared open vs. forced choice responses in two groups. To allow me to collect data from large numbers of subjects, I developed an experimental toolkit that uses a web-based crowdsourcing tool called Amazon Mechanical Turk (MTurk) and methods to collect and analyze data using MTurk. I collected data from 110 and 117 participants in the open-choice and forced-choice conditions respectively. I found that participants in the forced-choice condition were more likely to report the McGurk effect than the open-choice group (69% vs 42%, p = 10-7). This increase was consistent across all 8 stimuli. I showed that there was large variability in McGurk responses across subjects and stimuli for both open and forced choice conditions, ranging from 0% to 100% for subjects, and 30% to 80% for stimuli. In the second experiment, I attempted to influence the efficacy of McGurk stimuli by changing the speed of video playback. As technology becomes geared more towards audiovisual communication (e.g. videos on YouTube, Coursera), individuals now have the option of slowing information down or speeding them up to accommodate information processing needs. I modified the playback rate such that the stimuli were presented at .5x, 1x, and 2x speeds (slow, normal, fast) to 2 groups of participants (58 in one group and 60 in another) recruited using MTurk. I found that playback rate does indeed affect frequency of McGurk responses. Under slow speeds, McGurk responses dropped (an estimated 11%), while visual responses increased (12%), whereas, speeding up the video to 2x did not result in responses different from the normal speed (0.7%). The drop in McGurk responses in the slow condition may be explained with increase in onset asynchrony between the visual and auditory cues.en_US
dc.format.mimetypeapplication/pdfen_US
dc.identifier.citationBasu Mallick, Debshila. "Factors Affecting Audiovisual Speech Perception as Measured by the McGurk Effect." (2016) Diss., Rice University. <a href="https://hdl.handle.net/1911/96262">https://hdl.handle.net/1911/96262</a>.en_US
dc.identifier.urihttps://hdl.handle.net/1911/96262en_US
dc.language.isoengen_US
dc.rightsCopyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.en_US
dc.subjectAudiovisualen_US
dc.subjectspeech perceptionen_US
dc.subjectMcGurk effecten_US
dc.titleFactors Affecting Audiovisual Speech Perception as Measured by the McGurk Effecten_US
dc.typeThesisen_US
dc.type.materialTexten_US
thesis.degree.departmentPsychologyen_US
thesis.degree.disciplineSocial Sciencesen_US
thesis.degree.grantorRice Universityen_US
thesis.degree.levelDoctoralen_US
thesis.degree.nameDoctor of Philosophyen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
BASUMALLICK-DOCUMENT-2016.pdf
Size:
6.57 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
5.83 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
LICENSE.txt
Size:
2.6 KB
Format:
Plain Text
Description:
Download
Collections
Rice University Theses and Dissertations