Abstract: In daily lives, we often receive information from different modalities. Previous studies have found that multisensory stimuli can capture attention more effectively (i.e., multisensory enhancement) compared to unisensory stimuli. In the past decades, researchers have paid increasing attention to multisensory enhancement. However, it remains unclear whether the cognitive load affects the attention capture of multisensory stimuli. Some studies have shown that attention capture by multisensory stimuli is not influenced by the load, while others have demonstrated that attention capture by multisensory stimuli is modulated by the load. By adopting complex face and syllable stimuli, the present study aims to explore whether attention capture by audiovisual stimuli is modulated by the working memory load. Moreover, we are also interested in whether the multisensory enhancement for each modality is affected by the type of working memory load.
In present study, participants were required to perform a working memory task and a gender identification task, while verbal (in Experiment 1) or visual (in Experiment 2) working memory load was manipulated, respectively. Two within-participant variables were used: working memory load (low load vs. high load) and modality (visual, auditory vs. audiovisual). In Experiment 1, two or six numbers were presented at the center of the screen for 1000 ms, and participants were required to remember these numbers and their sequence. Then a visual (a face), auditory (a syllable), or a bimodal audiovisual target was presented for 700 ms, followed by a 1500 ms blank screen. Participants were instructed to judge the gender of the target as quickly and accurately as possible. At the end of each trial, a number in the memory set was presented and participants were asked to report the next number. In Experiment 2, spatial orientations were used in the working memory task instead of visual numbers, that is, participants had to remember the orientation of each line and the locations of the lines. Other experimental settings in Experiment 2 were the same as those in Experiment 1.
For the two experiments, a significant main effect of modality was found. Responses to multisensory targets were significantly faster than those to auditory targets, indicating that bimodal audiovisual stimuli could capture attention more effectively than unisensory stimuli. Our results support that, compared to unisensory auditory stimuli, bimodal audiovisual stimuli are more efficient for attention capture. Moreover, a significant interaction between working memory load and modality was found. That is, the crossmodal benefit was affected both by the working memory load and sensory modality. With the increase of verbal working memory load, the benefit of auditory stimuli from multisensory enhancement increased significantly, while the benefit of visual stimuli from multisensory enhancement remained unchanged. In contrast, with the increase of visual working memory load, the benefit of auditory stimuli from multisensory enhancement remained unchanged, while the benefit of visual stimuli from multisensory enhancement decreased significantly. These results suggest that working memory load affects the attention capture of multisensory stimuli, and the benefits of visual and auditory modalities from multisensory enhancement are modulated by the type of working memory load.

Key words: working memory load, multisensory enhancement, visual, auditory, attention

摘要： 本研究采用面孔和声音刺激,考察工作记忆负载对视听多通道刺激注意捕获的影响。实验1和实验2分别操纵言语和视觉工作记忆负载。结果发现相比单通道听觉刺激,双通道刺激能够更有效捕获注意。并且,视听多通道促进效应受到负载类型和通道类型的影响,表现为随着言语工作记忆负载升高,听觉从多通道促进的获益会升高,而视觉从多通道促进的获益没有显著变化。与之相反,随着视觉工作记忆负载升高,听觉从多通道促进的获益没有显著变化,视觉从多通道促进的获益则显著降低。结果支持视听多通道刺激的注意捕获受负载影响,且在不同负载类型下,视、听通道从多感觉促进效应的获益存在不同。

关键词: 工作记忆负载, 多感觉促进效应, 视觉, 听觉, 注意