|
nif:isString
|
-
These studies were approved by a University of Waterloo Research Ethics Committee. Participants signed and received separate consent forms for the mass testing and analytic thinking surveys, which were all completed online. All participants indicated their willingness to consent via button press. Participants received course credit for the mass testing and analytic thinking surveys.
Participants completed a series of reasoning problems and cognitive ability measures as part of a larger project on the everyday consequences of analytic thinking (see [2]). This includes, but is not limited to, religious belief. All participants were University of Waterloo students who signed up for a study on “thinking styles and reasoning” online through the participant pool. The only exclusion criteria was that participants had to have completed the department-wide mass-testing and pre-screen questionnaires prior to signing up for the thinking styles study (not every student in the participant pool completes one or both of these questionnaires). Participants were not permitted to sign up for the study more than once (i.e., someone who participated in Study 1 was not permitted to sign up for Studies 2–4 in the subsequent semesters). Demographic characteristics of the participants can be found in Table 1.
Table data removed from full text. Table identifier and caption: 10.1371/journal.pone.0153039.t001 Demographics for full participant samples (i.e., excluding those who failed the attention check) in Studies 1–4. a One participant did not indicate their gender.
We had complete data for 381 participants. At the end of the analytic thinking survey (but not in the mass testing or pre-screen surveys), participants were given an attention check question. The same instruction check was used for all four studies, with some slight variations (see below). For this, they were presented with a list of common activities and, in the instruction box, it read: “Below is a list of leisure activities. If you are reading this, please choose the ‘other’ box below and write ‘I read the instructions’.” Nine participants failed the instruction check and were excluded from subsequent analysis. This left us with 372 participants (see Table 1). We also asked participants if they had seen the CRT before and 77 participants (20.7%) responded affirmatively. Their data were retained but analyses are reported with and without these participants. Study 1 was completed in the winter term of 2013. A subset of the data from Study 1, 3, and 4 were previously published in an investigation of CRT scoring strategies [15]. For this, participants who completed the analytic thinking survey were permitted to complete a second survey with analytic thinking disposition measures and this subset of participants are included in Pennycook et al. [15]. In the current studies, the full sample is included and additional data from thinking disposition questionnaires is reported in S2 Text. The results for the thinking disposition questionnaires parallel our performance-based findings.
We had complete data for 158 participants. Nine participants failed the instruction check and were excluded from subsequent analysis. This left us with 149 participants (see Table 1), 20.7% (N = 31) of which responded affirmatively when asked if they have seen the CRT before. Study 2 was completed in the spring term of 2013.
We had complete data for 406 participants. The attention check was made more difficult for Studies 3 and 4 by inserting an introductory screen prior to the instruction check that said: “For the final part of the study, we are interested in the types of things that you do in your spare time”. This had a large effect: 127 participants failed and were excluded from subsequent analysis. This left us with 279 participants (see Table 1), 21.5% (N = 60) of which responded affirmatively when asked if they have seen the CRT before. Study 3 was completed in the fall term of 2013. Components of this data set were also previously published in an investigation of analytic thinking and smartphone use [9].
We had complete data for 398 participants. One hundred and thirty one participants failed the instruction check and were excluded from subsequent analysis. This left us with 267 participants (see Table 1), 25.1% (N = 67) of which responded affirmatively when asked if they have seen the CRT before. Study 4 was completed in the winter term of 2014.
All items can be found in S3 Text. A breakdown of the materials for each study can be found in Table 2. Base-rate neglect problems, used in Studies 1 and 2, were replaced with a longer heuristics and biases battery in Studies 3 and 4.
Table data removed from full text. Table identifier and caption: 10.1371/journal.pone.0153039.t002 Materials for Studies 1–4. CRT = Cognitive Reflection Test. a The additional CRT questions in Study 1 (N = 3) differed from those used in Studies 3 and 4 (N = 4; see S3 Text.
The religious belief scale from the mass testing survey included statements about eight conventional religious beliefs (Pennycook et al. [22], Study 2): heaven, hell, miracles, afterlife, angels, demons, soul, and the devil/Satan. Participants indicated their agreement/disagreement with the statements (where agreement meant that they held the belief in question) on the following 5-point scale: 1) I strongly disagree, 2) I disagree, 3) I don’t know, 4) I agree, 5) I strongly agree. The scale had good internal consistency: Cronbach’s α = 0.94 in each of the four studies and in the combined data set. Participants also indicated the ‘type’ of God that they believed in on a 7 point scale [4]: 1) A personal God, 2) God as an impersonal force, 3) A God who created everything, but does not intervene in human affairs [Deism], 4) Don’t know whether or not any Gods exist [Negative Agnostic], 5) Don’t know whether or not any Gods exist and no one else does either [Positive Agnostic], 6) I don’t believe in Gods of any sort [Negative Atheist], and 7) I believe that God does not exist [Positive Atheist]. To ease exposition, a theism measure was created by combining theists (options 1–3), agnostics (options 4 & 5), and atheists (options 6 & 7). For the religious affiliation question in the pre-screen survey, participants were presented with a list of religious affiliations and asked to select the option that they most strongly identified with. The list included the following options: Agnostic, Atheist, Baha’i, Buddhist, Chinese Traditional, Christian, Christian (specifically Catholic), Christian (specifically Protestant), Hindu, Humanist, Jewish, Muslim, No religion, Sikh, Taoist, and Other/not listed. Across the entire data set (i.e., Studies 1–4 combined), 13.3% (N = 142) of the sample selected agnostic, 12.5% (N = 133) selected atheist, 16% (N = 171) of the sample chose ‘no religion’, 42% (N = 448) of the sample selected one of the three Christian options, 14.8% (N = 149) chose a non-Christian religious affiliation. The remaining 2.2% (N = 24) did not provide a response and were excluded from the affiliation analysis.
Multiple measures of analytic cognitive style were included across the four studies. Each measure is intended to cue an incorrect intuitive response that requires additional analytic processing to override. The original 3-item CRT [14] was included in each study. Additional CRT items were added for Studies 1, 3, and 4. Three items were added for Study 1 and four items were added for Studies 3 and 4 [17]. In Studies 1 and 2, we also included 6 incongruent base-rate neglect problems [22]. These problems were intermixed with 6 congruent and 6 neutral problems. The following is an incongruent problem [31]: In a study 1000 people were tested. Among the participants there were 995 nurses and 5 doctors. Paul is a randomly chosen participant of this study. Paul is 34 years old. He lives in a beautiful home in a posh suburb. He is well spoken and very interested in politics. He invests a lot of time in his career. What is most likely? (a)Paul is a nurse. (b)Paul is a doctor.Base-rate neglect refers to the propensity for individuals to underweight or ignore the base-rate information (i.e., 995 nurses/5 doctors) in lieu of the more intuitive stereotypical information (i.e., Paul more closely resembles the stereotype of a doctor than a nurse). Incongruent problems contain a conflict between base-rate and stereotype (as above) whereas both sources of information suggest the same response for congruent problems. Neutral problems do not contain stereotypes in the personality description. The proportion of base-rate responses for incongruent problems has been shown to correlate negatively with religious belief in prior work [4, 22]. Finally, in Studies 3 and 4, a 14-item battery of heuristics and biases problems was administered [9, 10, 16, 17]. The battery included problems such as the conjunction fallacy and the gambler’s fallacy.
Measures of cognitive ability were included as control variables in each of the four studies. For this, a 12-item verbal intelligence test (the “Wordsum” [32]) was administered in each study. Participants also completed a 3-item numeracy test [33] in Studies 1, 3, and 4. This was increased to a 5-item test in Study 2 [34].
The religious belief scale was administered in online mass testing surveys with a number of different scales. These scales differed across the four studies (see S1 Text for a breakdown), but their order within the mass testing survey was always randomized.
The religious affiliation question always came after a set of demographic questions taken in an online pre-screen questionnaire.
Participants first completed the base-rate neglect/heuristics and biases problems (depending on which study, see Table 2), followed by numeracy and the CRT. The Wordsum was always administered last.
Participants had to complete the mass testing and pre-screen surveys before they were eligible to sign up for the primary analytic thinking survey. The vast majority of participants completed the analytic thinking survey on a different day than the mass testing and/or pre-screen surveys. However, a relatively small proportion did complete the analytic thinking survey on the same day as the mass testing (8.5% of the sample) or pre-screen (3.9% of the sample) surveys. Nonetheless, there was nothing linking the analytic thinking survey to any individual measure within the mass testing or pre-screen batteries and the results are essentially identical for participants who completed the surveys on the same day (see S4 Text). We therefore retained the full sample in the following analyses.
|
![[This page as RDF]](http://data.gesis.org/somesci/static/rdf-icon.gif){kind=icon}