|
?:linkContext
|
-
attributes or features of both objects. Rather, they focus on one object and its most accessible features --in Suppose, for example, that object X is characterized by features A to F, as shown in , whereas object Y is characterized by features D to K. When asked to compare X (the subject of comparison) to Y (the referent of comparison), judges would focus on features A to F and would determine if these features are also present in Y. This would result in a relative neglect of features G to K, which are not part of judges' representation of the subject of comparison. Conversely, when asked to compare Y to X, judges would focus on features D to K, resulting in a relative neglect of features A to C. This has two important implications. First, more features of a given stimulus are taken into account if the stimulus serves as the subject rather than the referent of comparison. In the above example, judges would consider six features of object X (namely A to F) when comparing X to Y, but only three features (namely D to F) when comparing Y to X. Second, and more important for our present purposes, judges' consideration of features is restricted by the features that characterize the subject of comparison, and features of the referent that are not shared by the subject of comparison are likely to go unnoticed. As a result, the comparative judgment is based on a different selection of features when X is compared to Y, rather than Y to X. This differential selection of features is likely to result in different evaluations. Whereas these process assumptions have been developed and tested in the domain of similarity judgments (see , for a more detailed theoretical discussion), applications have also been demonstrated in other areas , and the same feature selection processes should also apply to evaluative judgments of the type, "Is candidate A more or less suitable than Candidate B?"; "Is Tennis more or less exciting to watch than soccer?"; and so on. In each case, reversing the direction of comparison should influence which aspects judges focus on, and may hence result in different evaluations, in contrast to what formal logic would suggest. A number of different factors, such as the relative typicality or familiarity of the objects, may influence which object serves as the subject or as referent when individuals make a spontaneous comparison (see . The most important factor for survey researchers, however, is the wording of the question, namely, whether we ask respondents to compare X to Y, or Y to X. In the present paper, we report several experiments in which we explored the impact of question wording and the underlying cognitive processes. In a first experiment, conducted with 30 students at a German university, we tested the impact of reversing the direction of comparison on a variety of evaluative judgments. In one condition, we asked subjects to think of their high school teachers and to evaluate whether their female teachers were more or less empathetic than their male teachers, thus making the female teachers the subject of comparison. In the other condition, we asked whether the male teachers were more or less empathetic than the female teachers, thus making the male Comparative judgments 4 teachers the subject of comparison (see appendix for question wording). Likewise we asked subjects to compare the quality of life in Greece to that in Denmark, or the quality of life in Denmark to that in Greece. Similarly, they had to evaluate if tennis is more or less exciting to watch on TV than soccer, or if soccer is more or less exciting to watch than tennis. In the latter the case the direction of comparison was suggested by the response alternatives presented (see appendix). As shown in , the quality of life was judged to be higher in Denmark than in Greece, independent of the direction of comparison. But the advantage of Denmark over Greece was higher when subjects had to compare Greece to Denmark, rather than Denmark to Greece. Apparently taking more of Greece's unique features into account enhanced Denmark's superiority. The impact of the direction of comparison was even more dramatic in the other two examples. Here, reversing the direction of comparison did not merely influence the relative advantage of one object over the other, but reversed the ordinal position. When our respondents compared their female to their male teachers, they on average evaluated their female teacher as more empathetic than their male teachers. But when they compared their male to their female teachers, they evaluated their male teachers as more empathetic than their female teachers. The scope of the reversal is most evident when we dichotomize the scale, as shown in . When comparing female to male teachers, 41% of our respondents evaluated female teachers as more empathetic than male teachers but only 9% did so when they compared their male to there female teachers. In contrast, male teachers were seen as more empathetic Comparative judgments 5 by 55% of the respondents in the latter condition, but only 12% in the former. Similarly, tennis was judged as less exciting than soccer when respondents compared tennis to soccer (35% more exciting vs. 65% less exciting), but as more exciting than soccer when they compared soccer to tennis (77% more exciting vs. 15% less exciting), with hardly any undecided respondents. As these examples demonstrate, respondents' evaluations are strongly affected by the direction of comparison that is elicited by the specific wording of the question. Moreover, the emerging differences are not negligible. Depending on the specific case, different directions of comparison may even result in preference reversals, as was the case for the entertainment value of tennis and soccer. On theoretical grounds, we assume that the direction of comparison effects observed in above experiment reflect that subjects focus on the features of the subject of comparison, thereby neglecting unique features of the referent. However, our first experiment did not allow us to control which features subjects actually used. Accordingly, we could not predict if being the subject of comparison would result in a more positive evaluation (as was the case in the teacher example) or in a more negative evaluation (as was the case in the other examples). In general, a stimulus that has unique positive features should be evaluated more positively when it serves as the subject rather than the referent of the comparison. This follows from the assumption that more of its unique positive features are taken into account in the former than in the latter case. For the same reason, a stimulus that has unique negative features should be evaluated more negatively when it serves as the subject of the comparison. Hence, the valence of the unique features should determine the specific nature Comparative judgments 7 of the asymmetry that results from the direction of comparison elicited by the question. To test this prediction, we need to control the number and the valence of the features that respondents consider in making their judgment. To accomplish this, we conducted several subsequent experiments with students of the University of Mannheim (sample size varied between 28 and 106 in the different experiments) and asked subjects to evaluate previously unknown stimuli that we described to them. For example, in one study, subjects were asked to evaluate the suitability of two candidates for an executive position. Candidate A was described as possessing five positive and two neutral attributes, whereas Candidate B was described as possessing only three positive attributes and four neutral ones. Accordingly, Candidate B was clearly less qualified than Candidate A. However, according to our theoretical assumptions, subjects should be more likely to notice this difference in qualification when they compare Candidate A to Candidate B, than when they compare Candidate B to Candidate A. As shown in , the results provided strong support for this prediction. In fact, the actual superiority of Candidate A was only reflected in respondents' judgments when Candidate A served as the subject of comparison. In that case, they noticed that Candidate A had more positive features than Candidate B, resulting in a preference for Candidate A. In contrast, when respondents had to compare Candidate B to Candidate A, thus making B the subject of comparison, they missed the actual superiority of Candidate A and evaluated both candidates as equally qualified. In this case, Candidate B benefitted from being the Moderating Variables: If direction of comparison effects reflect a selective use of relevant features, we may expect that these effects are attenuated when judges are highly motivated. In that case, they may conduct a more complete information search, thus increasing the likelihood that features of the referent are considered as well. For example, if the judgment is very important one might be pondering an issue for a long time, literally comparing back and forth between stimuli. Certainly, comparing two job offers or two cars is likely to elicit more elaborate comparison efforts than comparing two breakfast cereals. Thus, a high "need for validity" , that is, a high motivation to form an accurate judgment, should reduce the observed asymmetry. We tested this assumption by manipulating subjects' motivation to form an accurate judgment in an experiment with 47 students of the University of Mannheim. For half of our subjects we increased the personal relevance of the task by telling them that being able to Comparative judgments 8 make accurate social judgments on the base of minimal information is a good predictor of social skills and social success. They were led to believe that they could compare themselves to previously established norms after the completion of the task. The other half of the subjects received no such instructions and the importance of the comparison judgment was down-played by pretending that the rating was used as a pilot test for another experiment. The results confirmed our predictions, as shown in . When the judgment was rendered personally relevant, the otherwise obtained direction of comparison effect was largely reduced. This suggests that the direction of comparison, elicited by question wording, may have little impact when respondents are highly motivated to form an accurate judgment. In that case, they may engage in a more elaborated comparison process that is not restricted by the features of the subject of comparison. motivation, we may expect that it is also affected by the placement of the question in the questionnaire. Research on question placement suggests that task involvement is higher in the beginning than at the end of an interview (cf. . We would thus expect higher asymmetries in comparative judgments when the comparison task is presented later in the questionnaire rather than at the beginning. We are currently testing this prediction and first results indicate support for more pronounced asymmetries in later questions rather than at the beginning of a questionnaire. In summary, our findings demonstrate that comparative judgments are strongly influenced by the specific wording of the question. When respondents are asked to compare X to Y, they focus on the features of X and check if these features are also present in Y. In doing so, they take more of the unique features of X into account than they would if they compared Y to X. Moreover, they neglect unique features of Y, which are not brought to mind by the features of X. As a result, comparisons of X to Y are based on a different selection of features than comparisons of Y to X, resulting in different evaluations. Moreover, the emerging differences are not negligible. At the extreme, the direction of comparison may reverse the ordinal position of two stimuli, as was the case in the evaluation of teachers' empathy or the entertainment value of tennis and soccer, assessed in our first study. Note, however, that the processes described here will not always result in different evaluations. For example, the specific constellation of the features of two stimuli may be such that both directions of comparison result in the use of the same features, or in the use of different features that have highly similar evaluative implications. In either case, no difference in evaluation would emerge as a function of the direction chosen. As another limiting condition, it is important to emphasize that the observed effects reflect the operation of a somewhat effortful feature comparison procedure. However, respondents may not always be involved enough to engage in the effort of recalling a Comparative judgments 10 number of specific features, or they may lack the relevant knowledge in the first place. In this case, they may base their judgment on some cue or single feature that allows for an easy evaluation. For example, they may evaluate a political candidate on the basis of his party What are the implications of our findings for survey practice? Given that the direction of comparison makes a difference, how should a comparative question be worded? In the ideal case, the comparison question should follow the direction of comparison that people are likely to choose spontaneously in daily life. Unfortunately, we do not yet fully understand what determines if a given object is spontaneously chosen as the subject or the referent of comparison. On theoretical grounds, we may assume that the more familiar or salient of two stimuli is likely to serve as subject, but this effect may easily be overridden by the nature of the context. For example, in comparing candidates for a presidential election, one may conjecture that the incumbent is more salient and accessible than his competitors, and therefore likely to serve as the subject of comparison. Hence, respondents may be more likely to spontaneously compare Bush to Clinton rather than Clinton to Bush. But if the preceding questions focused on Clinton, thus rendering him temporarily more accessible, this tendency may easily reverse. And how about Perrot? Does the attention that he currently receives in the media make it more likely that he is compared to Bush, rather As another alternative, one may consider the use of undirected question wording. This, however, poses other problems. On the one hand, it is unclear how a comparative question can be worded in a way that does not suggest a direction of comparison. Obviously, we may ask respondents to compare "Bush and Clinton", rather than "Bush 1q Clinton", but Thus, what should we do? Instead of conceptualizing direction of comparison effects as source of undesirable bias, it seems more fruitful to utilize the underlying cognitive dynamics. In many cases, the question we try to answer by assessing a comparative judgment calls for a specific direction of comparison anyway. For instance, in deciding whether to launch a product innovation or not, the crucial information is how the innovation is evaluated in comparison to its predecessor, but not how the predecessor is evaluated in comparison to the new product which is to replace it. The latter comparison is one that consumers are unlikely to face. Similarly, if we want to know how people evaluate social change we should ask them to compare the present to the past, rather than the past to the present. This follows from the observation that most spontaneous comparisons over time are triggered by salient aspects of one's current experiences, thus rendering the present the subject of comparison (see , for a more detailed discussion). As usual, the most suitable strategy is to analyze the problem that we want to learn about and to word the question accordingly. The more we understand the cognitive processes that underlie respondents' judgments, in surveys as well as in natural contexts, the more we will be able to use them to our advantage. We close by noting that the wording effect addressed in the present paper has not been documented in the survey literature. Given that formal logic as well as common sense suggest that comparing X to Y should result in the same conclusions as comparing Y to X, it is not surprising that survey methodologists saw no need to explore the potential impact of the direction of comparison requested from respondents. Rather, cognitive research into the nature of similarity judgments identified processing strategies that Comparative judgments 13 implied that a wording effect of the type observed here had to exist, if one only looked for it. We emphasize this point because most of the recent applications of cognitive theories to survey measurement provided theoretical analyses of response effects that had long been documented by survey methodologists. In contrast, the present line of research indicates that cognitive theories may also allow us to identify response effects that have so far gone unnoticed. If this is good or bad news for survey research is not for us to decide. soccer is more exciting to watch than tennis, or that soccer is less exciting to watch than tennis? ( ) Soccer is much more exciting than tennis ( ) Soccer is more exciting than tennis ( ) Soccer is somewhat more exciting than tennis ( ) Soccer and tennis are equally exciting ( ) Soccer is somewhat less exciting than tennis ( ) Soccer is less exciting than tennis ( ) Soccer is much less exciting than tennis Detecting and identifying change: Additions versus deletions Journal of Experimental Psychology: Human Perception and Performance The influence of unique features and direction of comparison on preferences Journal of Experimental Social Psychology The effects of fatigue on judgments of interproduct similarity International Journal of Research in Marketing Lay epistomo-logic -process and contents: Another look at attribution theory Psychological Review The effect of direction of comparison on the selection of causal explanations Journal of Experimental Social Psychology Die Vergleichsrichtung bestimmt das Ergebnis von Vergleichsprozessen: Ist-Idealdiskrepanzen in der Beziehungsbeurteilung Zeitschrift für Sozialpsvcholopie Subjective assessments and evaluations of change: Lessons from social cognition research The role of attribute knowledge and overall evaluations in comparative judgment Organizational Behavior and Human Decision Processes General principles and individual difference in the self as a habitual reference point: An examination of self-other judgments of similarity Social Cognition Asking questions. A practical guide to questionnaire design Features of similarity Psychological Review Studies of similarity Cognition and categorization Schuessler Methoden der Analyse zeitbezogener Daten. Vortragsskripten der ZUMA-Arbeitstagung vom 25 ALLBUS Manfred Küchler Verallgemeinerte Lineare Modelle in der empirischen Sozialforschung Glenn R. Carroll Dynamic analysis ot discrete dependent variables: A didactic essay Manfred Küchler Zur Messung der Stabilität von Wahlerpotentialen Manfred Küchler Zur Konstanz der Recalltrage Rolf Porst " ALLBUS 1982" -Systematische Vanablenübersicht und erste Ansätze zu einer Kritik des Fragenprogramms Peter Ph. Mohler SAR -Simple AND Retrieval mit dem Siemens-EDT-Textmanipulationsprogramm Cornelia Krauth Vergleichsstudien zum ALLBUS Bernd Wegener Two approaches to the analysis of judgments ot prestige: Interindivi dual differences and the general scale Rolf Porst Synopse der ALLBUS-Variablen. Die Systematik des ALLBUS-Fragenprogramms und ihre inhaltliche Ausgestaltung Mohler Qualshop (ZUMA-Arbeitstagung zum ' ' Datenmanagement bei qualitativen Erhebungsverfahren") -Sammlung von Arbeitspapieren und -berichten, Teil I + II Bernd Wegener Gibt es Sozialprestige? Günter Rothe Ein Ansatz zur Konstruktion inferenzstatistisch verwertbarer Indices Norbert Schwarz Assessing frequency reports ot mundane behaviors: Contributions of cognitive psychology to questionnaire construction Norbert Schwarz, B. Scheurmg (sub.) Judgments of relationship satisfaction: Inter-and intramaividual Anmerkungen zur Siemens BS2000 Version Michael Schneid Datenerhebung am PC -Vergleich der Interviewprogramme "interv * ' und "THIS" Norbert Schwarz, Bettina Scheuring Die Vergleichsrichtung bestimmt das Ergebnis von Vergleichsprozessen: Ist -Idealdiskrepanzen m der Partnerwahrnehmung Norbert Schwarz, Bettina Scheuring Die Vergleichsrichtung bestimmt das Ergebnis von Vergleichs prozessen Allgemeine Bevölkerungsumtrage der Sozialwissenschaften -ALLBUS 1988 -Norbert Schwarz Feelings as Information: Informational and Motivational Functions of Affective States Günter Rothe Jackknife and Bootstrap: Resampling-Verfahren zur Genauigkeitsschätzung von Parameterschätzungen Herbert Bless Eine deutsche Version des "Mood Survey" von Underwood und Froming Ulrich Mueller Evolutionary Fundamentals of Social Inequality, Dominance and Cooperation Robert Huckfeldt Noncompliance and the Limits ot Coercion: The Problematic Enforcement of Unpopular Laws Peter Ph. Mohler Dirk Wagner The Salience of Comparison Standards and tne Activation or Social Norms: Consequences ror Judgments of Happiness and their Communication Jutta Kreiselmaier, Rolf Porst Methodische Probleme bei der Durcntührung telelomscher Befragungen: Stichprobenziehung und Ermittlung von Zielpersonen Rainer Mathes Modulsystem und Netzwerktechnik. Neuere inhaltsanalytische Verfahren zur Analyse von Kommunikationsmhaiten Elisabeth Noelle-Neumann Response Order Effects in Dichotomous Questions: The Impact of Administration Mode Norbert Schwarz Elisabeth Noelle-Neumann, Thomas Munjtel Response Order Effects m Long Lists: Primacy, Recency, and Asymmetric Contrast Effects Herbert Bless Constructing Reality and Its Alternatives: An Inclusion / Exclusion Model of Assimilation and Contrast Ettects in Social Judgment Herbert Bless Bärbel Knäuper, E. Tory Higgms Der Einfluß von Rangordnungsaufgaben auf nachfolgende Denkprozesse: Zur Aktivierung prozeduraler Sets Bettina Scheuring, Norbert Schwarz Selbstberichtete Verhaltens-und Symptomhautlgkeiten: Was Befragte aus Antwortvorgaben des Fragebogens lernen Norbert Schwarz, Herbert Bless Scandals and the Public's Trust in Politicians: Assimilation and Contrast Ettects Rolf Porst Auställe und Verweigerungen bei einer teletomscnen Befragung Uwe Blien, Heike Wirth, Michael Müller Identification risk tor microüata stemming from ofncial statistics Petra Beckmann Methodological Report ISSP Michael Wiedenbeck Methodenbericht zur Allgemeinen Bevölkerungsumtrage der Sozialwissenschaften Oded Löwenbein Einkommensanalysen auf der Grundlage amtlicher Daten und Umfragedaten: Ergebnisse zur betrieblichen Seniontät und Arbeitslosigkeit ISSP Basic Information on the ISSP Data Collection Norbert Schwarz In weicher Reihenfolge fragen? GROBID - A machine learning software for extracting information from scholarly documents attributes or features of both objects. Rather, they focus on one object and its most accessible features --in Tversky's (1977) Suppose, for example, that object X is characterized by features A to F, as shown in Figure 1 , whereas object Y is characterized by features D to K. Figure 1 When asked to compare X (the subject of comparison) to Y (the referent of comparison), judges would focus on features A to F and would determine if these features are also present in Y. This would result in a relative neglect of features G to K, which are not part of judges' representation of the subject of comparison. Conversely, when asked to compare Y to X, judges would focus on features D to K, resulting in a relative neglect of features A to C. This has two important implications. First, more features of a given stimulus are taken into account if the stimulus serves as the subject rather than the referent of comparison. In the above example, judges would consider six features of object X (namely A to F) when comparing X to Y, but only three features (namely D to F) when comparing Y to X. Second, and more important for our present purposes, judges' consideration of features is restricted by the features that characterize the subject of comparison, and features of the referent that are not shared by the subject of comparison are likely to go unnoticed. As a result, the comparative judgment is based on a different selection of features when X is compared to Y, rather than Y to X. This differential selection of features is likely to result in different evaluations. Comparative judgments 3 Whereas these process assumptions have been developed and tested in the domain of similarity judgments (see Tversky, 1977; Tversky & Gati, 1978 , for a more detailed theoretical discussion), applications have also been demonstrated in other areas (Agostinelli, Sherman, Fazio & Hearst, 1986; Houston, Sherman, Baker, 1990; Sanbonmatsu, Kardes & Gibson, 1991; Srull & Gaelick, 1983; Schwarz & Scheuring, 1989; McGill, 1990) , and the same feature selection processes should also apply to evaluative judgments of the type, "Is candidate A more or less suitable than Candidate B?"; "Is Tennis more or less exciting to watch than soccer?"; and so on. In each case, reversing the direction of comparison should influence which aspects judges focus on, and may hence result in different evaluations, in contrast to what formal logic would suggest. A number of different factors, such as the relative typicality or familiarity of the objects, may influence which object serves as the subject or as referent when individuals make a spontaneous comparison (see Tversky, 1977) . The most important factor for survey researchers, however, is the wording of the question, namely, whether we ask respondents to compare X to Y, or Y to X. In the present paper, we report several experiments in which we explored the impact of question wording and the underlying cognitive processes. Does the Direction of Comparison Make a Difference? In a first experiment, conducted with 30 students at a German university, we tested the impact of reversing the direction of comparison on a variety of evaluative judgments. In one condition, we asked subjects to think of their high school teachers and to evaluate whether their female teachers were more or less empathetic than their male teachers, thus making the female teachers the subject of comparison. In the other condition, we asked whether the male teachers were more or less empathetic than the female teachers, thus making the male Comparative judgments 4 teachers the subject of comparison (see appendix for question wording). Likewise we asked subjects to compare the quality of life in Greece to that in Denmark, or the quality of life in Denmark to that in Greece. Similarly, they had to evaluate if tennis is more or less exciting to watch on TV than soccer, or if soccer is more or less exciting to watch than tennis. In the latter the case the direction of comparison was suggested by the response alternatives presented (see appendix). Figure 2 As shown in Figure 2 , the quality of life was judged to be higher in Denmark than in Greece, independent of the direction of comparison. But the advantage of Denmark over Greece was higher when subjects had to compare Greece to Denmark, rather than Denmark to Greece. Apparently taking more of Greece's unique features into account enhanced Denmark's superiority. The impact of the direction of comparison was even more dramatic in the other two examples. Here, reversing the direction of comparison did not merely influence the relative advantage of one object over the other, but reversed the ordinal position. When our respondents compared their female to their male teachers, they on average evaluated their female teacher as more empathetic than their male teachers. But when they compared their male to their female teachers, they evaluated their male teachers as more empathetic than their female teachers. The scope of the reversal is most evident when we dichotomize the scale, as shown in Figure 3 . When comparing female to male teachers, 41% of our respondents evaluated female teachers as more empathetic than male teachers but only 9% did so when they compared their male to there female teachers. In contrast, male teachers were seen as more empathetic Comparative judgments 5 by 55% of the respondents in the latter condition, but only 12% in the former. Similarly, tennis was judged as less exciting than soccer when respondents compared tennis to soccer (35% more exciting vs. 65% less exciting), but as more exciting than soccer when they compared soccer to tennis (77% more exciting vs. 15% less exciting), with hardly any undecided respondents. As these examples demonstrate, respondents' evaluations are strongly affected by the direction of comparison that is elicited by the specific wording of the question. Moreover, the emerging differences are not negligible. Depending on the specific case, different directions of comparison may even result in preference reversals, as was the case for the entertainment value of tennis and soccer. Why Does it Make a Difference? On theoretical grounds, we assume that the direction of comparison effects observed in above experiment reflect that subjects focus on the features of the subject of comparison, thereby neglecting unique features of the referent. However, our first experiment did not allow us to control which features subjects actually used. Accordingly, we could not predict if being the subject of comparison would result in a more positive evaluation (as was the case in the teacher example) or in a more negative evaluation (as was the case in the other examples). In general, a stimulus that has unique positive features should be evaluated more positively when it serves as the subject rather than the referent of the comparison. This follows from the assumption that more of its unique positive features are taken into account in the former than in the latter case. For the same reason, a stimulus that has unique negative features should be evaluated more negatively when it serves as the subject of the comparison. Hence, the valence of the unique features should determine the specific nature Comparative judgments 6 Comparative judgments 7 of the asymmetry that results from the direction of comparison elicited by the question. To test this prediction, we need to control the number and the valence of the features that respondents consider in making their judgment. To accomplish this, we conducted several subsequent experiments with students of the University of Mannheim (sample size varied between 28 and 106 in the different experiments) and asked subjects to evaluate previously unknown stimuli that we described to them. For example, in one study, subjects were asked to evaluate the suitability of two candidates for an executive position. Candidate A was described as possessing five positive and two neutral attributes, whereas Candidate B was described as possessing only three positive attributes and four neutral ones. Accordingly, Candidate B was clearly less qualified than Candidate A. However, according to our theoretical assumptions, subjects should be more likely to notice this difference in qualification when they compare Candidate A to Candidate B, than when they compare Candidate B to Candidate A. Figure 4 As shown in Figure 4 , the results provided strong support for this prediction. In fact, the actual superiority of Candidate A was only reflected in respondents' judgments when Candidate A served as the subject of comparison. In that case, they noticed that Candidate A had more positive features than Candidate B, resulting in a preference for Candidate A. In contrast, when respondents had to compare Candidate B to Candidate A, thus making B the subject of comparison, they missed the actual superiority of Candidate A and evaluated both candidates as equally qualified. In this case, Candidate B benefitted from being the Moderating Variables: Respondent Motivation and Fatigue If direction of comparison effects reflect a selective use of relevant features, we may expect that these effects are attenuated when judges are highly motivated. In that case, they may conduct a more complete information search, thus increasing the likelihood that features of the referent are considered as well. For example, if the judgment is very important one might be pondering an issue for a long time, literally comparing back and forth between stimuli. Certainly, comparing two job offers or two cars is likely to elicit more elaborate comparison efforts than comparing two breakfast cereals. Thus, a high "need for validity" (Kruglanski, 1980) , that is, a high motivation to form an accurate judgment, should reduce the observed asymmetry. We tested this assumption by manipulating subjects' motivation to form an accurate judgment in an experiment with 47 students of the University of Mannheim. For half of our subjects we increased the personal relevance of the task by telling them that being able to Comparative judgments 8 make accurate social judgments on the base of minimal information is a good predictor of social skills and social success. They were led to believe that they could compare themselves to previously established norms after the completion of the task. The other half of the subjects received no such instructions and the importance of the comparison judgment was down-played by pretending that the rating was used as a pilot test for another experiment. Figure 6 The results confirmed our predictions, as shown in Figure 6 . When the judgment was rendered personally relevant, the otherwise obtained direction of comparison effect was largely reduced. This suggests that the direction of comparison, elicited by question wording, may have little impact when respondents are highly motivated to form an accurate judgment. In that case, they may engage in a more elaborated comparison process that is not restricted by the features of the subject of comparison. If the size of direction of comparison effects varies as a function of respondents' motivation, we may expect that it is also affected by the placement of the question in the questionnaire. Research on question placement suggests that task involvement is higher in the beginning than at the end of an interview (cf. Sudman & Bradbum, 1983; Johnson, Lehman & Home, 1990) . We would thus expect higher asymmetries in comparative judgments when the comparison task is presented later in the questionnaire rather than at the beginning. We are currently testing this prediction and first results indicate support for more pronounced asymmetries in later questions rather than at the beginning of a questionnaire. Conclusions In summary, our findings demonstrate that comparative judgments are strongly influenced by the specific wording of the question. When respondents are asked to compare X to Y, they focus on the features of X and check if these features are also present in Y. In doing so, they take more of the unique features of X into account than they would if they compared Y to X. Moreover, they neglect unique features of Y, which are not brought to mind by the features of X. As a result, comparisons of X to Y are based on a different selection of features than comparisons of Y to X, resulting in different evaluations. Moreover, the emerging differences are not negligible. At the extreme, the direction of comparison may reverse the ordinal position of two stimuli, as was the case in the evaluation of teachers' empathy or the entertainment value of tennis and soccer, assessed in our first study. Note, however, that the processes described here will not always result in different evaluations. For example, the specific constellation of the features of two stimuli may be such that both directions of comparison result in the use of the same features, or in the use of different features that have highly similar evaluative implications. In either case, no difference in evaluation would emerge as a function of the direction chosen. As another limiting condition, it is important to emphasize that the observed effects reflect the operation of a somewhat effortful feature comparison procedure. However, respondents may not always be involved enough to engage in the effort of recalling a Comparative judgments 10 number of specific features, or they may lack the relevant knowledge in the first place. In this case, they may base their judgment on some cue or single feature that allows for an easy evaluation. For example, they may evaluate a political candidate on the basis of his party What are the implications of our findings for survey practice? Given that the direction of comparison makes a difference, how should a comparative question be worded? In the ideal case, the comparison question should follow the direction of comparison that people are likely to choose spontaneously in daily life. Unfortunately, we do not yet fully understand what determines if a given object is spontaneously chosen as the subject or the referent of comparison. On theoretical grounds, we may assume that the more familiar or salient of two stimuli is likely to serve as subject, but this effect may easily be overridden by the nature of the context. For example, in comparing candidates for a presidential election, one may conjecture that the incumbent is more salient and accessible than his competitors, and therefore likely to serve as the subject of comparison. Hence, respondents may be more likely to spontaneously compare Bush to Clinton rather than Clinton to Bush. Comparative j udgments 11 But if the preceding questions focused on Clinton, thus rendering him temporarily more accessible, this tendency may easily reverse. And how about Perrot? Does the attention that he currently receives in the media make it more likely that he is compared to Bush, rather As another alternative, one may consider the use of undirected question wording. This, however, poses other problems. On the one hand, it is unclear how a comparative question can be worded in a way that does not suggest a direction of comparison. Obviously, we may ask respondents to compare "Bush and Clinton", rather than "Bush 1q Clinton", but Thus, what should we do? Instead of conceptualizing direction of comparison effects as source of undesirable bias, it seems more fruitful to utilize the underlying cognitive dynamics. In many cases, the question we try to answer by assessing a comparative judgment calls for a specific direction of comparison anyway. For instance, in deciding whether to launch a product innovation or not, the crucial information is how the innovation is evaluated in comparison to its predecessor, but not how the predecessor is evaluated in comparison to the new product which is to replace it. The latter comparison is one that consumers are unlikely to face. Similarly, if we want to know how people evaluate social change we should ask them to compare the present to the past, rather than the past to the present. This follows from the observation that most spontaneous comparisons over time are triggered by salient aspects of one's current experiences, thus rendering the present the subject of comparison (see Schwarz, Bless, & Wanke, 1992 , for a more detailed discussion). As usual, the most suitable strategy is to analyze the problem that we want to learn about and to word the question accordingly. The more we understand the cognitive processes that underlie respondents' judgments, in surveys as well as in natural contexts, the more we will be able to use them to our advantage. We close by noting that the wording effect addressed in the present paper has not been documented in the survey literature. Given that formal logic as well as common sense suggest that comparing X to Y should result in the same conclusions as comparing Y to X, it is not surprising that survey methodologists saw no need to explore the potential impact of the direction of comparison requested from respondents. Rather, cognitive research into the nature of similarity judgments (Tversky, 1977) identified processing strategies that Comparative judgments 13 implied that a wording effect of the type observed here had to exist, if one only looked for it. We emphasize this point because most of the recent applications of cognitive theories to survey measurement provided theoretical analyses of response effects that had long been documented by survey methodologists. In contrast, the present line of research indicates that cognitive theories may also allow us to identify response effects that have so far gone unnoticed. If this is good or bad news for survey research is not for us to decide. soccer is more exciting to watch than tennis, or that soccer is less exciting to watch than tennis? ( ) Soccer is much more exciting than tennis ( ) Soccer is more exciting than tennis ( ) Soccer is somewhat more exciting than tennis ( ) Soccer and tennis are equally exciting ( ) Soccer is somewhat less exciting than tennis ( ) Soccer is less exciting than tennis ( ) Soccer is much less exciting than tennis terminology the subject of comparison. Having determined the key features of the subject of comparison, judges check to what extent the other object, called the referent, possesses these features as well. However, by focusing on the features of the subject of comparison, judges are likely to miss unique features of the referent of comparison. Figure 3 3 Figure 3 subject of comparison because focussing on his positive features resulted in a relative neglect of the additional unique positive features of Candidate A. As shown in Figure 5, this systematic impact of the direction of comparison elicited by question wording replicated over a wide variety of different stimulus materials and specific experimental procedures. Figure 5 5 In combination, these findings demonstrate that judges focus on the features that characterize the subject of.comparison, and make less use of the features that characterize the referent of the comparison. Accordingly, the direction of comparison determines which features respondents draw on in forming a judgment. As a result, comparing A to B may lead to a different evaluation than comparing B to A, in contrast to what formal logic would require. Our data indicates that the size of direction of comparison effects increases as respondents' motivation decreases. How motivated respondents are, is in part a function of the relevance of the judgment at hand and of respondent fatigue that may develop over the Comparative judgments 9 course of a lengthy interview. By the same token, other factors that interfere with a more elaborate comparison process, such as increased time pressure, may also increase the impact o f the direction of comparison elicited by the wording of the question. membership without considering any of his other attributes. If so, the direction of comparison would show little effect. In general, direction of comparison effects require a certain amount of knowledge about the attributes of the stimuli and a certain amount of effort in comparing them. A lack of knowledge or effort shortcuts the underlying processes and eliminates the expected effects. At the other extreme, a high degree of motivation may also eliminate the effect, as it may lead respondents to consider all available attributes, as we have seen in the experiment reported above. These considerations imply that direction of comparison effects should be most pronounced when respondents have some degree of knowledge about the relevant features and are moderately motivated to engage in their retrieval and comparison. than Bush to him? At the present stage of research, we simply do not know. Moreover, different respondents may potentially use different politicians as the subject of comparison, reflecting who is more salient to them at the time of judgment. Hence, the recommendation that question wording should reflect the direction of comparison that respondents are likely to use spontaneously is solid on theoretical grounds, but difficult to implement. What, however, are the alternatives? On first glance, it seems that one may vary the direction of comparison and may pool the obtained responses to avoid systematic bias. However, an inspection of the tennis/soccer results shown in Figures 2 and 3 reveals that this solution is less than convincing. As this example illustrates, the average value that results from pooling both comparison questions may not reflect the ordinal information provided by either judgment and may conceal important differences. Nevertheless, varying the direction of comparison has the advantage that it draws attention to possible differences and discourages the overinterpretation of the outcome of one single direction of comparison. chances are that what we introduce first is still likely to serve as subject rather than as referent. More important, the successful implementation of an a priori undirected question, if possible, results in a variant of the averaging strategy discussed above, as different Comparative judgments 12 respondents will use different directions of comparison, depending on individual knowledge or salience differences --but we lost information about the impact of the direction chosen. Appendix Example Question Wordings A) Comparison between female and male teachers Female -male comparison: Thinking of your teachers in high school, would you say that the female teachers were more empathetic with regard to academic and personal problems than the male teachers, or were they less empathetic? Female teachers were less empathetic 12345678 9 more empathetic Male -female comparison: Think of your teachers in high school, would you say that the male teachers were more empathetic with regard to academic and personal problems than the female teachers, or were they less empathetic? Male teachers were less empathetic 12345678 9 more empathetic Comparative judgments 16 To compare the relative evaluations across question wordings, the scale for one condition was reverse coded. B) Comparison between tennis and soccer Tennis -soccer comparison: Tennis and soccer are the two sports that draw the largest audience. When you compare both sports with regard to how entertaining they are for a TV audience, would you say that tennis is more exciting to watch than soccer, or that tennis is less exciting to watch than soccer? ( ) Tennis is much more exciting than soccer ( ) Tennis is more exciting than soccer ( ) Tennis is somewhat more exciting than soccer ( ) Tennis and soccer are equally exciting ( ) Tennis is somewhat less exciting than soccer ( ) Tennis is less exciting than soccer ( ) Tennis is much less exciting than soccer Soccer -tennis comparison: Soccer and tennis are the two sports that draw the largest audience. When you compare both sports with regard to how entertaining they are for a TV audience, would you say that Detecting and identifying change: Additions versus deletions G Agostinelli S J Sherman R H Fazio E S Hearst Journal of Experimental Psychology: Human Perception and Performance 12 The influence of unique features and direction of comparison on preferences D A Houston S J: Sherman S M Baker Journal of Experimental Social Psychology 25 The effects of fatigue on judgments of interproduct similarity M D Johnson D R Lehman D R Home International Journal of Research in Marketing 7 Lay epistomo-logic -process and contents: Another look at attribution theory A W Kruglanski Psychological Review 87 The effect of direction of comparison on the selection of causal explanations A L Mcgill Journal of Experimental Social Psychology 26 Die Vergleichsrichtung bestimmt das Ergebnis von Vergleichsprozessen: Ist-Idealdiskrepanzen in der Beziehungsbeurteilung N Schwarz B Scheuring Zeitschrift für Sozialpsvcholopie 20 Subjective assessments and evaluations of change: Lessons from social cognition research N Schwarz H Bless M Wänke ZUMA-Arbeitsbericht. 92/12 The role of attribute knowledge and overall evaluations in comparative judgment D M Sanbonmatsu F R Kardes B D Gibson Organizational Behavior and Human Decision Processes 48 General principles and individual difference in the self as a habitual reference point: An examination of self-other judgments of similarity T K Srull L Gaelick Social Cognition 2 Asking questions. A practical guide to questionnaire design S Sudman N M Bradburn Jossey-Bass San Francisco Features of similarity A Tversky Psychological Review 4 Studies of similarity A Tversky I ; Gati N J Hillsdals Cognition and categorization E. Rosch & B. B. Lloyd Erlbaum Schuessler Methoden der Analyse zeitbezogener Daten. Vortragsskripten der ZUMA-Arbeitstagung vom 25 Gerhard Armmger Willibald Nagl F Karl 09. -05.10.79 Erika Brückner Hans-Peter Kirschner Roli Porst Peter Pruler ALLBUS Peter Schmidt Methodenbericht zum Manfred Küchler Verallgemeinerte Lineare Modelle in der empirischen Sozialforschung Glenn R. Carroll Dynamic analysis ot discrete dependent variables: A didactic essay Manfred Küchler Zur Messung der Stabilität von Wahlerpotentialen Manfred Küchler Zur Konstanz der Recalltrage Rolf Porst "ALLBUS 1982" -Systematische Vanablenübersicht und erste Ansätze zu einer Kritik des Fragenprogramms Peter Ph. Mohler SAR -Simple AND Retrieval mit dem Siemens-EDT-Textmanipulationsprogramm Cornelia Krauth Vergleichsstudien zum Manfred Küchler Thomas P Wilson Don H Zimmerman Integration Von Qualitativen Und Quantitativen Forschungsansätzen Gerhard Arminger Horst Busse ALLBUS Bernd Wegener Two approaches to the analysis of judgments ot prestige: Interindivi dual differences and the general scale Rolf Porst Synopse der ALLBUS-Variablen. Die Systematik des ALLBUS-Fragenprogramms und ihre inhaltliche Ausgestaltung Werner Hagstotz Hans-Peter Kirscnner Rolt Porst im ALLBUS 1980 und ALLBUS 1982 Peter Prüfer Methodenbericht zum "ALLBUS Mohler Qualshop (ZUMA-Arbeitstagung zum ' ' Datenmanagement bei qualitativen Erhebungsverfahren") -Sammlung von Arbeitspapieren und -berichten, Teil I + II Bernd Wegener Gibt es Sozialprestige? Manfred Küchler Peter Ph Konstruktion und Validitat der Magnitude-Prestige-Skala Günter Rothe Ein Ansatz zur Konstruktion inferenzstatistisch verwertbarer Indices Ute Hauck Norbert Schwarz Assessing frequency reports ot mundane behaviors: Contributions of cognitive psychology to questionnaire construction Norbert Schwarz, B. Scheurmg (sub.) Judgments of relationship satisfaction: Inter-and intramaividual Anmerkungen zur Siemens BS2000 Version Michael Schneid Datenerhebung am PC -Vergleich der Interviewprogramme "interv * ' und "THIS" Norbert Schwarz, Bettina Scheuring Die Vergleichsrichtung bestimmt das Ergebnis von Vergleichsprozessen: Ist -Idealdiskrepanzen m der Partnerwahrnehmung Norbert Schwarz, Bettina Scheuring Die Vergleichsrichtung bestimmt das Ergebnis von Vergleichs prozessen Ist-Idealdiskrepanzen in der Beziehungsbeurteilung Norbert Schwarz, George F. Bishop, Hans-J. Hippier Fritz Strack Psychological Sources Of Response Effects in Self-Administered And Telephone Surveys Allgemeine Bevölkerungsumtrage der Sozialwissenschaften -ALLBUS 1988 -Norbert Schwarz Feelings as Information: Informational and Motivational Functions of Affective States Günter Rothe Jackknife and Bootstrap: Resampling-Verfahren zur Genauigkeitsschätzung von Parameterschätzungen Herbert Bless Michael Braun Reiner Trometer Michael Wiedenbeck Methodenbericht Gerd Bohner Norbert Schwarz und Fritz Strack Happy and Mindless Eine deutsche Version des "Mood Survey" von Underwood und Froming Ulrich Mueller Evolutionary Fundamentals of Social Inequality, Dominance and Cooperation Robert Huckfeldt Noncompliance and the Limits ot Coercion: The Problematic Enforcement of Unpopular Laws Peter Ph. Mohler Gerd Bohner Norbert Schwarz Und Stefan E Hormuth Die Stimmungs-Skala Katja Frehsen und Ute Hauck CUI -Computerunterstützte Inhaltsanalyse Grundzüge und Auswahlbibliographie zu neueren Anwendungen Cornelia Züll, Peter Ph. Mohler Der General Inquirer III - Dirk Wagner The Salience of Comparison Standards and tne Activation or Social Norms: Consequences ror Judgments of Happiness and their Communication Jutta Kreiselmaier, Rolf Porst Methodische Probleme bei der Durcntührung telelomscher Befragungen: Stichprobenziehung und Ermittlung von Zielpersonen Fritz Strack Norbert Schwarz Brigitte Chassein Dieter Kern Ausschöpfung und Nonresponse Qualität der Daten Rainer Mathes Modulsystem und Netzwerktechnik. Neuere inhaltsanalytische Verfahren zur Analyse von Kommunikationsmhaiten Jutta Kreiselmaier Peter Prüfer Margrit Rexroth Der Interviewer im Pretest Elisabeth Noelle-Neumann Response Order Effects in Dichotomous Questions: The Impact of Administration Mode Norbert Schwarz -J Hans Norbert Hippier Schwarz -J Hans Hippier Elisabeth Noelle-Neumann, Thomas Munjtel Response Order Effects m Long Lists: Primacy, Recency, and Asymmetric Contrast Effects Herbert Bless Constructing Reality and Its Alternatives: An Inclusion / Exclusion Model of Assimilation and Contrast Ettects in Social Judgment Herbert Bless P Joseph F Roland Gerd Fellhauer Bohner Bärbel Knäuper, E. Tory Higgms Der Einfluß von Rangordnungsaufgaben auf nachfolgende Denkprozesse: Zur Aktivierung prozeduraler Sets Bettina Scheuring, Norbert Schwarz Selbstberichtete Verhaltens-und Symptomhautlgkeiten: Was Befragte aus Antwortvorgaben des Fragebogens lernen Norbert Schwarz, Herbert Bless Scandals and the Public's Trust in Politicians: Assimilation and Contrast Ettects Rolf Porst Auställe und Verweigerungen bei einer teletomscnen Befragung Uwe Blien, Heike Wirth, Michael Müller Identification risk tor microüata stemming from ofncial statistics Petra Beckmann Methodological Report ISSP Forgas Mood Effects on Interpersonal Preferences: Evidence for Motivated Processing Strategies Joseph P. Forgas Affective Influences on Interpersonal Perception Norbert Schwarz Michael Wiedenbeck Methodenbericht zur Martina Wasmer Achim Koch Allgemeinen Bevölkerungsumtrage der Sozialwissenschaften Allbus Oded Löwenbein Einkommensanalysen auf der Grundlage amtlicher Daten und Umfragedaten: Ergebnisse zur betrieblichen Seniontät und Arbeitslosigkeit Uwe Blien ISSP Basic Information on the ISSP Data Collection Petra Beckmann Peter Möhler Koit Uher Norbert Schwarz In weicher Reihenfolge fragen? Ellen D Riggle C Victor Robert S Ottati Jr James Wyer Kuklmski Norbert Schwarz
(xsd:string)
|
![[This page as RDF]](https://data.gesis.org/gesiskg/static/rdf-icon.gif){kind=icon}