The effect of object processing in content-dependent source memory.

BACKGROUND: Previous studies have suggested that the study condition of an item influences how the item is encoded. However, it is still unclear whether subsequent source memory effects are dependent upon stimulus content when the item and context are unitized. The present fMRI study investigated the effect of encoding activity sensitive to stimulus content in source memory via unitization. In the scanner, participants were instructed to integrate a study item, an object in either a word or a picture form, with perceptual context into a single image.
RESULTS: Subsequent source memory effects independent of stimulus content were identified in the left lateral frontal and parietal regions, bilateral fusiform areas, and the left perirhinal cortex extending to the anterior hippocampus. Content-dependent subsequent source memory effects were found only with words in the left medial frontal lobe, the ventral visual stream, and bilateral parahippocampal regions. Further, neural activity for source memory with words extensively overlapped with the region where pictures were preferentially processed than words, including the left mid-occipital cortex and the right parahippocampal cortex.
CONCLUSIONS: These results indicate that words that were accurately remembered with correct contextual information were processed more like pictures mediated by integrated imagery operation, compared to words that were recognized with incorrect context. In contrast, such processing did not discriminate subsequent source memory with pictures. Taken together, these findings suggest that unitization supports source memory for both words and pictures and that the requirement of the study task interacts with the nature of stimulus content in unitized source encoding.

Background

Neural correlates of successful memory formation have been investigated with the functional magnetic resonance imaging (fMRI) subsequent memory procedure, in which neural activity during encoding for later remembered items is compared with activity for later forgotten items. Subsequent memory effects are often expressed as enhanced encoding activity for remembered items in several brain regions, such as the left prefrontal cortex, the fusiform area, and the medial temporal lobe (MTL) [1-3]. Further, subsequent memory tends to reveal the influence of the study condition in which the item is experienced at the cortical level. Seminal behavioral studies of memory have suggested that memory representation results from how an item is processed at encoding [4-6]. From this perspective, episodic memory is the by-product of study processing based on the circumstances in which an event is experienced. In accordance with this theoretical principle, previous fMRI studies of subsequent memory effects have shown that cortical encoding activity for successful memory of an item depends upon the nature of the study task, the modality of the study item, and the material of the study item [2,7-9].

In these studies, the differences in study conditions, including study task (semantic vs. non-semantic) or stimulus content such as modality (visual vs. auditory) and material (verbal vs. pictorial), dissociated the cortical regions where neural activity for successful memory formation was identified. For example, subsequent memory of words was predicted in the left inferior prefrontal cortex and the left fusiform region, whereas encoding activity for pictures was identified in the right inferior prefrontal cortex and bilateral fusiform areas [2]. The MTL effects of memory formation also showed left lateralized activation for words and right lateralized activation for pictures and faces [9]. Semantic encoding of successfully remembered words was related with activity in the left and medial prefrontal regions; however, non-semantic subsequent memory effects were found in the right prefrontal cortex, bilateral intra-parietal sulci and bilateral fusiform gyri [7]. Moreover, subsequent memory effects for visually presented words were reported in the right fusiform cortex, while auditory subsequent memory effects were identified in bilateral superior temporal sulci [10].

However, relatively little is known about whether the formation of associations, as opposed to encoding of single items, also exhibits study condition-specific effects of encoding. The extant findings of task-dependent effects in memory associations are less consistent. For example, a study that compared task effects on associative memory of word pairs using semantic and phonological tasks found that extensive activity in the left inferior prefrontal cortex predicted word pairs that were later successfully remembered as an intact pair in both encoding tasks; task-specific effects, however, were negligible [11]. Another study that compared semantic versus perceptual study conditions for associative memory of word pairs reported that the same left inferior prefrontal regions were involved in subsequent associative memory effects common to both conditions; however, perceptual associative encoding effects were also found in the left temporo-occipital and bilateral parietal cortices as well as the right parahippocampal cortex [12].

Other studies of subsequent source memory (item-source associations) showed that neural correlates of source memory differed depending upon study task or stimulus content [13,14]. Subsequent source memory of words studied with a semantic task was localized in the left temporo-occipital cortex, while successful source memory effects with a non-semantic encoding task were found in the right temporo-occipital region, revealing hemispheric differences between subsequent source memory effects that depend upon the study task [13]. Subsequent source memory effects also varied by stimulus content, such that source encoding activity for words were found in the left inferior prefrontal cortex, while successful source memory effects for objects were identified in the left perirhinal cortex [14].

In addition to these cortical effects, the MTL is known to be critical for forming associations [12,15,16]. From the memory process view, the MTL subregions are specialized for different memory processes. That is, the hippocampus and the parahippocampal cortex are integral to the formation of associations based on recollection, whereas the perirhinal cortex tends to support familiarity-based item recognition [17-21]. Functional dissociation within the MTL for different memory processes has been supported by findings that increased activity in the hippocampus predicts subsequent recognition based on recollection (e.g., source memory) but not familiarity [22-24], whereas familiarity-based judgments (e.g., item memory) have been related with increased perirhinal activity during encoding [18,22,23]. It was further proposed that the perirhinal cortex could support source memory based on familiarity through unitization of an item and context [25]. As opposed to the conventional way of binding (i.e., non-unitized encoding), in which separate item and context are to be combined through associations, unitization is assumed to bind context to the item as an item feature. From this view, source memory through unitization may be supported by familiarity. Further, source memory via unitization was supported by perirhinal activity, which is consistent with the proposal of the perirhinal cortex for familiarity-based source recognition [25,26].

It has also been suggested that MTL subregions are involved in encoding of heterogeneous classes of stimulus content [22,27]. First, the hippocampus is involved in processing of content-independent binding. For instance, encoding activity in the anterior hippocampus was identified for recollection of both objects and scenes, reflecting the content-independent role of the hippocampus [28]. Other MTL subregions selectively engage in processing of different types of content. Specifically, the parahippocampal cortex is selective to visuo-spatial processing such as scene processing [29-33]. The perirhinal cortex is involved in object processing, and the role of perirhinal cortex in object encoding was found across different recognition judgments (i.e., Remember, Know, and New) and source memory [29]. Similarly, a MTL subsequent memory study that examined source encoding with objects and scenes reported the dissociation of the MTL cortex by content: the perirhinal cortex contributed to source encoding for objects, but the parahippocampal cortex supported source encoding of scenes [32].

Though previous studies suggested content-dependent encoding effects, most of those studies examined content-dependent effects with non-unitized encoding tasks or focused on MTL activity. As alluded to earlier, unitization of an item and context may change the way that the item and context are bound; therefore, unitized source memory may be supported by different encoding activity at the cortical level. Considering that the memory process involved in unitized encoding may be different from the memory process for non-unitized encoding, an interesting question is whether subsequent memory effects specific to stimulus content would appear with unitized encoding both at the cortical and MTL levels. This is important, as it investigates how encoding activity specific to each trial (i.e., stimulus content) is influenced by study processing across trials (i.e., encoding operation) in forming associations. Previous studies that used non-unitized encoding tasks reported content-independent effects in the hippocampus and content-dependent effects in the cortical areas. However, it is not clear whether unitized encoding is sensitive to content-dependent subsequent memory effect. Additionally, given that non-unitized encoding tends to elicit hippocampal activity for source memory formation across different types of content, it is of interest whether unitized encoding would show similar MTL source memory effects across verbal and pictorial content. In what follows, we report a study of the formation of source memory for words and pictures via unitization.

Participants were presented a list of study items (i.e. words and pictures) on a colored background and were instructed to imagine the item in the background color. This meant that participants actively had to generate a mental presentation for words but not necessarily for pictures. Thus, imagery-based operations required by the study task would have different effects depending on whether the content was verbal or pictorial. By comparing encoding activity of words and pictures varied by later source memory performance (source correct vs. source incorrect), we investigated the effect of imagery-based operations in source memory by stimulus content.

Previous studies of subsequent memory effects have reported encoding activity for perceptual processing of visually-presented stimuli in the parietal and occipital cortices, as well as the fusiform cortex [10,13,34-36]. Further, perirhinal activity has been reported for the formation of source memory with objects through unitization [29,31,32]. Thus, it is likely that parietal/occipital activity as well as perirhinal activity would contribute to source encoding of both types of content. On the other hand, participants had to generate a visuo-spatial representation of the word and place it in the color context for words, whereas pictures were already presented in the necessary visual form. That said, visuo-spatial processing and binding would be critical for forming successful source memory with words by engaging in the study task, but less so with pictures. Then, neural activity for visuo-spatial processing and binding would distinguish words with contextual information from words without correct context to a greater extent than such activity with pictures, which would emerge as content-dependent subsequent source memory effects. Further, as pictures were presented in the pictorial form during study, the brain regions that showed picture-preferential processing (pictures > words) regardless of later source memory effects would be the areas where subsequent source memory effects selective to words would be identified.

As such, we hypothesized the following for the present experiment:

(1) For subsequent source memory effects independent of stimulus content, unitization of item and color would be supported by encoding of visually presented objects, as evidenced by increased activity in the parieto-occipital cortices, the fusiform gyrus, and the perirhinal cortex, in addition to the established role of hippocampal activity in source memory formation overall.

(2) Study requirement for engaging in visuo-spatial processing and binding for source memory formation would affect words more than pictures, and this would call for greater subsequent source memory effects selective to words in the ventral visual pathway and the parahippocampal cortex.

(3) Subsequent source memory effects for words would indicate that these words were processed more like pictures, based on the overlap of regions where pictures were preferentially processed compared to words such as parietal and occipital regions.

Results and discussion

Behavioral results

Proportions of recognition judgments (source correct, source incorrect, item miss) by stimulus content (word, picture) are displayed in Table 1. The proportions of correct source judgments for both stimulus contents were significantly greater than the chance level (.25), ts [23] > 15.82, ps < .001. There was no main effect of stimulus content, but there was a significant interaction between content and recognition judgment, F[2,46] = 14.42, p < .001. Follow-up tests showed that more source correct judgments were made with words than pictures, t[23] = 2.34, p < .05, whereas more source incorrect judgments were made with pictures than words, t[23] = 5.26, p < .001, although item misses were greater with words than pictures, t[23] =3.09, p < .005. Further, using d’ as a measure of recognition accuracy, more accurate source judgments were made for words (1.73 [.15]) than for pictures (1.27 [.17]), t[23] = 4.28, p < .001.

Table 1

Mean test response proportions (SEM) and mean study reaction times (in ms) as a function of stimulus content and memory judgment

Stimulus content	Studied			New
	Source correct	Source incorrect	Miss	Correct rejection	False alarm
Word	.69 (.03)	.19 (.02)	.12(.02)	.91 (.03)	.09 (.03)
	1656 (84)	1671 (80)	1661 (83)
Picture	.66 (.02)	.26 (.02)	.08 (.01)	.93 (.01)	.07 (.01)
	1675 (84)	1733 (83)	1767 (74)

Study reaction times are also shown in Table 1, segregated by stimulus content and later source memory judgment. A 2 × 2 ANOVA with stimulus content (word, picture) and source judgment (source correct, source incorrect, item miss) as factors was conducted on these data. The ANOVA showed the main effect of stimulus content, indicating longer reaction times for pictures than for words, F[1,23] = 8.59, p < .005. However, there was neither a main effect of source judgment nor an interaction between stimulus content and source judgment on study times.

fMRI Results

Analyses of subsequent source memory effects were based on contrasts between encoding activity of study items that were later accurately endorsed with the studied color (source correct) as opposed to encoding activity of study items that were later recognized but with an incorrect color (source incorrect). Item misses were not included in the analysis due to an insufficient number of trials.

Neural activity for source memory formation independent of stimulus content

First, we sought neural activity associated with the formation of source memory independent of stimulus content via unitization. In order to find subsequent source memory effects, the statistical parametric mappings (SPMs) for the main effect of source memory across stimulus content (source correct > source incorrect) were exclusively masked with the SPMs for the interaction of source memory by content. The outcome of this analysis is shown in Table 2. Source encoding activity was localized exclusively in the left hemisphere, including superior to inferior lateral parts of the prefrontal cortex, the orbito-frontal region, the lateral parietal and occipital cortices, and the fusiform gyrus (Figure 1A). Source memory independent of stimulus content was also identified in the left perirhinal cortex extending to the left hippocampus (Figure 1B). The mean parameter estimates extracted from all of these clusters demonstrated that encoding activity was greater for source correct than for source incorrect items, and these differences were independently reliable for both stimulus content from all clusters (ps < .05). The contrast in the opposite direction (source incorrect > source correct exclusively masked with the interaction) did not reveal any suprathreshold clusters.

Table 2

Subsequent source memory effects independent of stimulus content

Coordinates (x y z)			Z (# of voxels)		Region	BA
Non-MTL effects
−12	48	42	4.42	104	L superior frontal gyrus	8/9
−42	9	36	4.30	56	L inferior frontal gyrus	44
−9	9	−21	4.25	39	L inferior orbito-frontal cortex	47
−42	−81	33	4.47	156	L angular gyrus/Inferior/superior parietal lobule/ Mid occipital cortex	19/39
−54	−57	−21	3.52	39	L fusiform gyrus	37
MTL effects
−21	−15	−24	3.84	39	L perirhinal cortex/anterior hippocampus

Figure 1

Content-independent subsequent source memory effects. A: Lateral content-independent subsequent source memory effects. B: MTL content-independent effects overlaid on mean anatomical image across subjects. Mean parameter estimates elicited by stimulus content (sc: source correct, si: source incorrect, ps < .05).

Neural activity for source memory formation depending on stimulus content

Next, we looked for encoding activity for source memory varied by the content of the stimulus through integrated encoding of item and context. Subsequent source memory effects dependent on content were identified by exclusively masking source memory effects in one stimulus content type (e.g., source correct > source incorrect with word trials and vice versa) with subsequent source memory effects in the alternate content type (e.g., source correct > source incorrect with picture trials and vice versa). This procedure revealed only subsequent source memory effects selective to words, which occurred in the superior medial parts of the left frontal lobe, the right insula, and the left ventral visual stream (Table 3). As illustrated in Figure 2, MTL source memory effects selective to words were identified in bilateral middle/posterior parahippocampal cortices, and activity in the right parahippocampal cortex extended to the vicinity of the right hippocampus. ANOVAs on parameter estimates extracted from all clusters showing word-dependent source memory effects revealed significant interactions between stimulus content and source memory judgment, Fs[1,23] > 5.8, ps < .05. Follow-up tests revealed that words that were subsequently remembered with correct source information showed greater encoding activity in these regions than words recognized with incorrect source information (ps < .001), whereas pictures did not differ by source memory in these regions. Subsequent source memory effects selective to pictures did not reveal a single cluster over the threshold.

Table 3

Subsequent source memory effects selective to words

Coordinates (x y z)			Z (# of voxels)		Region	BA
Non-MTL effects
−15	36	30	3.59	27	L medial frontal gyrus	32
−24	27	54	4.25	57	L middle frontal gyrus	8
27	3	24	4.06	40	R insula/caudate
−3	−3	15	3.64	41	Thalamus
−42	−57	−18	3.86	49	L fusiform gyrus	37
−42	−69	9	4.27	77	L middle occipital gyrus	39
MTL effects
−33	−27	−21	3.61	17	L parahippocampal cortex	36
27	−33	−12	4.39	57	R parahippocampus/ hippocampus

Figure 2

Word-dependent subsequent source memory effects in the MTL. Effects are overlaid on a section (y = −33) of the across-subject mean anatomical image.

Overlap between subsequent source memory effects and processing effects of stimulus content

We also examined whether subsequent source memory effects depending on content overlapped with brain regions where corresponding stimulus content trials elicited more activity than alternate content trials. Subsequent source memory effects for one content type (e.g., source correct > source incorrect for words and vice versa) were inclusively masked with corresponding effects for content processing (e.g., words > pictures and vice versa). No overlap was found either with word or picture source memory effects in the area where words had elicited more activity than pictures. Rather, subsequent source memory effects for both words and pictures did overlap with brain regions where pictures had elicited more activity than words (Figure 3). Source encoding activity for pictures was overlapped with picture-preferential processing areas in the bilateral fusiform areas and the left superior parietal lobe. Extensive overlap was found between word source encoding activity and picture processing in bilateral parahippocampal cortices and occipital regions.

Figure 3

Overlap between content-dependent subsequent source memory effects and content-preferential processing areas. A: Cortical regions where subsequent source memory effects elicited by words (blue) and pictures (red) overlap with regions where activity is greater on picture trials than word trials (thresholded at p < .001, yellow). Areas of overlap between words and pictures are shown in purple. B: Overlap between word-dependent source memory effects and picture processing in bilateral parahippocampal cortices.

Discussion

The present study investigated content-dependent source memory effects by means of an integrated imagery operation. Successful source memory that did not differ between the two stimulus content types was predicted in the left inferior frontal gyrus (LIFG), the lateral parietal/occipital lobes, and the fusiform gyrus. In addition, MTL subsequent memory effects independent of stimulus content were identified in the left perirhinal cortex extending to the left anterior hippocampus. The formation of source memory selective to words recruited activity in the left frontal cortex, the ventral visual stream, and the bilateral parahippocampal cortices extending to the vicinity of the right posterior hippocampus. Picture-dependent subsequent source memory effects did not emerge over threshold. Finally, subsequent source memory effects for words extensively overlapped with the areas where pictures were preferentially processed.

Content-independent subsequent source memory effects were identified in a number of cortical regions. First, subsequent source memory effects were evident in the LIFG extending to the superior frontal regions. LIFG effects have been found in the formation of associative memory [11,12,37,38] and source memory [14,34] with both semantic and non-semantic encoding tasks. The current finding of LIFG involvement in integrated source encoding is consistent with the proposal that the LIFG is crucial for the formation of associations regardless of the type of study processing [11,38]. Encoding activity for source memory independent of stimulus content was also found in the left lateral parietal cortex, spanning from the angular gyrus to the superior parietal/occipital regions. These regions, especially the intra-parietal sulcus, are known to engage in goal-directed attentional processing [39,40], multi-modal processing [35,41], and perceptual processing [35,36,42], in addition to encoding of both item and source memories [7,10,18,43]. Previous studies also reported subsequent source memory effects in the LIFG and the left parietal regions with non-unitized encoding tasks. Thus, the present findings of the LIFG and the left parietal activity through integrated binding of item and context indicates that neural activity in these regions supports the formation of successful source memory mediated by perceptual processing through unitized encoding as well as non-unitized encoding.

In parallel with these cortical effects, content-independent subsequent source memory effects were identified in the left perirhinal cortex extending to the left anterior hippocampus, which is consistent with previous reports of perirhinal activity for source memory formation with objects [14,29,33] and prior findings of hippocampal activity for source encoding in general [12,31,32]. The formation of associations via unitization of an item and its contextual details is known to be supported by perirhinal activity, as the unitized representation of the item with context as an object can be recognized based on familiarity [44,45]. The current finding of perirhinal activity in source encoding with both words and pictures extends previous findings of perirhinal involvement in associations [18,19,32,33,46] in that source memory for both verbal and pictorial content is supported by perirhinal activity through unitization. Further, activity in the hippocampus has been found in subsequent source memory in a content-general manner, which confirms the established role of the hippocampus in memory associations overall [19,28,31].

While investigating content-dependent subsequent source memory effects, we found only word-dependent effects. These effects were evident in the visual ventral stream, the region critical for object identification (i.e., the “what” pathway), which suggests that accurate source memory of words was mediated by visuo-spatial object processing more so than inaccurate source memory of words. That is, words that were subsequently remembered with source information tended to be the ones with more object-like processing during encoding, whereas words that were not accompanied with correct source information tended to be the ones that were not visualized well enough for imagery operation to occur at a level comparable to those that elicited source correct judgments.a However, the level of object processing did not distinguish subsequent memory of pictures between source correct versus source incorrect. Considering that pictures were presented in the visual object form during encoding, object processing would likely occur for all pictures regardless of whether they were accompanied with correct contextual information or not, hence the smaller impact of object processing in subsequent source memory effects with pictures. This interpretation is supported by the significant interaction between stimulus content and source judgment revealing the differences in parameter estimates only with source correct words versus source incorrect words (see Results). In addition, insular activity findings complement previous findings of insular involvement in multi-modal processing of stimuli [42] as well as source encoding [34,47].

Word-dependent source memory effects were also identified in the bilateral parahippocampal cortices, extending to the vicinity of the right posterior hippocampus. These MTL effects reflect that activity of word-dependent source memory exhibits not only extensive object processing but also additional visuo-spatial and contextual processing for the words that were later remembered with correct source, compared with words without correct source, consistent with prior reports of parahippocampal activity for both spatial and nonspatial contexts [46,48]. On the other hand, there was no overlap between word-dependent source memory effects and the region where words were preferentially processed than pictures. Prior studies have shown overlap of content-dependent subsequent memory effects in the areas where corresponding content are preferentially processed [10,38,43]. The current, seemingly contradictory findings illustrate that successfully remembered words with accurate context are indeed the outcome of encoding in more of an ‘object’ manner due to the requirement of the study task. These findings suggest that adopting pictorial processing by engaging in the imagery-based encoding task was crucial for the formation of successful source memory only with words due to the interaction between the nature of study processing and the stimulus content.b In sum, the present findings demonstrate that the requirement of trial-general study processing influences the way that trial-specific stimulus content is experienced. Importantly, the present findings show that unitization is supported for different types of stimulus content.

Conclusions

We investigated neural activity specific to stimulus content via integrated source encoding with words and pictures. As predicted, subsequent source memory effects were evident in the left lateral frontal and parietal cortex as well as the left perirhinal cortex extending to the hippocampus, which is indicative of the significance of these regions in source memory formation for integrated object-context. Content-dependent source memory effects were found only with words in the ventral visual stream and the parahippocampal cortex, which reflects more visuo-spatial processing and contextual binding with words for successful source memory due to the requirement of imagery encoding operation. Collectively, the present findings demonstrate the effect of trial-general study processing in subsequent source memory effects depending on stimulus content through unitization.

Methods

Subjects

Twenty-four subjects participated in the experiment (16 females; 18–29 years). They were recruited from the University of Texas−Arlington community and compensated for their participation. All subjects reported being right−handed, being native English speakers, and having no history of neurological disease. Informed consent was obtained prior to participation. The experiment was approved by the University of Texas−Arlington and University of Texas Southwestern Medical Center Institutional Review Boards.

Materials

The experimental stimulus pool consisted of 175 concrete words (4–9 letter nouns) and 175 line-drawings depicting nameable objects. Three-hundred critical items were selected from the stimulus pool to construct the study and test lists. For each subject, a study list of 200 critical items (100 words and 100 line-drawing pictures) was created from the pool by random selection of items without replacement. Critical items were pseudo-randomly assigned to one of four color backgrounds (blue, green, red, or yellow), 25 words, and 25 pictures for each color. All words were presented in black, all pictures were presented in black-white line-drawing form, and both words and pictures were overlaid on the colored background. Each study list comprised an intermixed presentation of critical items with 100 null trials interspersed among the experimental trials. A test list was consisted 200 studied items intermixed with 100 new items (50 words and 50 pictures). Test items were presented on a gray background. Both study and test list sequences were constrained by no more than four consecutive presentations of a type of content or background color. An additional 50 items, 25 words, and 25 pictures were used for practice trials. The composition and ordering of the study and test lists were made anew for each subject.

All items were projected onto a screen viewed by subjects via a mirror mounted on the scanner head. For the study list, critical items were presented on one of four color backgrounds, while null events were presented on a gray background. For the test list, all items were presented on the gray background. All items were presented at a maximum visual angle of 7.1° × 7.1°.

Procedure

Subjects were given instructions and practice for experimental tasks prior to the experiment proper. The experiment consisted of two study-test cycles. For each study phase, a fixation cross was presented for 200 ms as a warning signal for the upcoming item, followed by a study item superimposed on the colored background for 2000 ms. The study item was replaced by a response prompt (‘+’) for 1300 ms prior to the onset of the next trial. Stimulus onset asynchrony was 3500 ms. Subjects were instructed to form an image of the object (e.g., ‘UMBRELLA’) in the background color (e.g., ‘RED’) and to make a pleasantness judgment for the colored object (‘RED UMBRELLA’). Note that the pleasantness judgment was made based on the mental image that integrated object and color. This study instruction was aimed to facilitate unitization of the object and color context. Subjects indicated their judgments by pressing a button with the index or middle finger of their right hand. The finger assignment to each response was counterbalanced across subjects. Four colors (blue, green, red, or yellow) were used for backgrounds, and 50 items (half words and half pictures) were presented in each colored background. For null trials, a fixation character (‘+’) was continuously displayed for 3500 ms on a gray background, and no response was required.

The test phase followed approximately 5 min after the end of the study phase. Test items were presented on the gray background. The content of the stimulus was kept consistent between study and test (e.g., if a stimulus was studied in a word form, it was also tested in the word form). Subjects were asked to judge whether the test item was previously studied in the experiment and, if it had been studied, to indicate in which color it was presented; subjects responded using a one-step source recognition response. Each study and test session lasted about 20 minutes.

fMRI scanning

A Philips Achieva 3T MR scanner (Philips Medical Systems, Andover, MA) fitted with a 32-channel RF receiver head coil was used to acquire both T1-weighted anatomical volume images (256 × 238 matrix, 1 mm3 voxels) and T*2-weighted echo-planar images (EPIs) (80 × 80 matrix, 3 mm × 3 mm in-plane resolution, axial acquisition, flip angle 70°, TE 30 ms) per volume. Each EPI volume comprised thirty 3 mm-thick axial slices acquired in a descending sequential order and separated by 1 mm, providing coverage of almost the entire brain. Data were acquired during the study and test phases in four scanning sessions comprising 270 volumes each, with a repetition time (TR) of 2 s. Five additional volumes were collected at the beginning of each run, but these were discarded to allow for T1 equilibration. The 3.5 s SOA allowed for an effective sampling rate of the hemodynamic response of 2 Hz.

fMRI data analysis

Data preprocessing and statistical analyses were performed with Statistical Parametric Mapping (SPM 8, Wellcome Department of Cognitive Neurology, London, UK: http://www.fil.ion.ucl.ac.uk), implemented in MATLAB 9 (Mathworks, Natick, MA). For each subject, functional images were registered to the first image of each scan session and temporally realigned to the middle slice using sinc interpolation. The resulting data were normalized to a standard EPI template based on the Montreal Neurological Institute (MNI) reference brain and resampled into 3 mm isotropic voxels using nonlinear basis functions [49]. The normalized images were smoothed with an isotropic 8 mm full-width half-maximum Gaussian kernel. The time series in each voxel were high-pass filtered to 1/128 Hz to remove low-frequency noise and scaled to a grand mean of 100 across both voxels and scans. T1-weighted anatomical images were coregistered to the mean EPI volume and normalized to a standard T1 template of the MNI brain. After normalization, an across-subject mean anatomical image was created as the study-specific template for identifying brain regions.

Statistical analysis was performed on the study phase data using a two-stage mixed effects model for investigating encoding activity associated with content-dependent source memory effects through unitization of the item and context. In the first stage, neural activity was modeled by delta functions at stimulus onset. The event-related blood oxygen-level dependent (BOLD) response was modeled by convolving these neural functions with a canonical hemodynamic response function (HRF) and its temporal and dispersion derivatives. In addition, six regressors were modeled for movement-related variance, and session-specific constant terms were employed to model the mean image intensity in each session. Parameter estimates for events of interest were measured for each subject using a General Linear Model. Non-sphericity of the error covariance was accommodated by an AR(1) model in which the temporal autocorrelation was estimated by pooling over suprathreshold voxels [50]. The parameters for each covariate and the hyperparameters governing the error covariance were estimated using Restricted Maximum Likelihood (ReML). In the second stage, linear contrasts of these subject-specific parameter estimates were computed, treating subjects as a random effect.

For the analysis of subsequent source memory effects by stimulus content, four events of interest were defined: ‘word-source correct’ (studied words that were recognized with correct study color); ‘word-source incorrect’ (studied words that were recognized albeit with incorrect study color); ‘picture-source correct’ (studied pictures that were accurately judged with study color); and ‘picture-source incorrect’ (studied pictures that were responded with incorrect color). All other study trials including item misses (studied items that were incorrectly judged as new) and no responses were modeled as events of no-interest, due to an insufficient number of trials.c

At the whole brain level, only effects surviving a voxel-wise threshold of p < .001 and a corrected cluster-wise threshold of p < .05 (greater than 16 voxels) were interpreted. When exclusive masking was applied to identify voxels where effects were not shared between two contrasts, the mask threshold was set at a two-tailed threshold of p < .1. Note that the more liberal the threshold of an exclusive mask, the more conservative the masking procedure. The threshold for the inclusive mask was set to p < .01. The peak voxels of clusters exhibiting reliable effects are reported in MNI coordinates. For the regions identified from main analyses, region-specific parameter estimates were extracted by peak voxel activity of a cluster for each subject and subjected to group-level statistical tests. The significance level for the region-specific parameter estimates was set to p < .05.

Endnotes

aWe thank Reviewer for clarification of this interpretation.

bAt the MTL level, subsequent source memory effects dependent on words may be accounted by recollection. That is, words that were recognized with correct source information recruited more activity related with recollection process in the parahippocampal cortex and the hippocampus due to binding of the item and context, compared with words without source memory. To put it differently, source correct words were more likely the words that were accompanied with the recollection process of binding the item and context compared with source incorrect words, whereas subsequent source memory dependent on pictures was not significantly distinguished by recollection process. We thank Reviewer for this suggestion.

cOnly 7 out of 24 participants had sufficient numbers of trials (> 10) for both words and pictures.

Competing interest

The authors declare that they have no competing interests.

Authors’ contributions

HP designed the study, participated in data collection, performed the statistical analysis, and wrote the manuscript. FL and CS participated in data collection. CA participated in writing the manuscript. All authors read and approved the manuscript.