Hup1804.vp

HUMAN PERFORMANCE, 18(4), 331–341Copyright 2005, Lawrence Erlbaum Associates, Inc.
The key to success in business is money and people. Personality psychology isabout people—it’s about the nature of human nature. Some understanding of hu-man nature—and the ability to measure its key components—would seem to offera huge advantage to applied psychologists. Despite its practical significance, per-sonality has lived a troubled existence in academic psychology. The topic was pop-ular after World War II. But the 1960s brought the response set (or faking) contro-versy, which challenged the foundation of personality assessment. Then the 1970sand 1980s brought Mischel’s revolution, which taught us that we can’t measurepersonality because it doesn’t exist—people’s actions are not determined by theirpersonalities but by “situational factors.” Situational factors are like dark matter inPhysics—they are strange, undefined, invisible forces that exist “out there,” thatcapture us, and then make us do their bidding.
In the 1990s, personality made a comeback in industrial psychology. The come- back was fueled by the news that well-constructed measures of personality predictjob performance almost as well as measures of cognitive ability, but with no ad-verse impact. The critics went silent for about 10 years, but now they are back (cf.
Schmitt, 2004). They argue that claims for the validity of personality measureshave been vastly overstated, that the data reveal only trivial relations between thesemeasures and occupational performance. The point of this article is to try to put thevarious issues in perspective and show that well-constructed measures of personal-ity are an indispensable tool for applied psychologists. The article is organized interms of four sections: (a) What is wrong with personality psychology? (b) What is This article is based on articles presented by Robert Hogan at the 32nd International Congress on Assessment Center Methods, October, 2004, Las Vegas, NV.
Correspondence should be sent to Robert Hogan, Hogan Assessment Systems, 2622 East 21st Street, Tulsa, OK 74144. E-mail: [email protected] wrong with the critics of personality psychology? (c) What is personality psychol-ogy? (d) What do the data actually say? WHAT IS WRONG WITH PERSONALITY PSYCHOLOGY? No one is more aware of the problems of personality psychology than I am. I cansummarize these problems in terms of three points. First, there are only about 200personality psychologists in the United States, and not many of them are active re-searchers. Among the active researchers, there is virtually no agreement regardingan agenda for the discipline. One group studies the sources of psychopathology. Asecond group tries to identify the basic structure of personality, with no concern forpractical applications. A third group evaluates the neuropsychological foundationsof personality and the heritability of the major dimensions. A fourth and very tinygroup, most of whom live in Oklahoma, is concerned with predicting importantpractical outcomes—competence, effectiveness, leadership, creativity, integrity.
Overall, however, there is no consensus about an intellectual agenda for the field,which means that the cumulative impact of the few existing researchers is mini-mized. In my view, the field only stays alive because it is intrinsically appealing tothe general public.
A second problem with personality psychology is a generalized lack of concern for measurement validity. There are perhaps 2,500 test publishers in the UnitedStates, and only a very few pay attention to validity. The furor caused by the recentvolume, The Cult of Personality, by Annie Murphy Paul (2004), is due entirely tothe willingness of publishers to sell tests with no demonstrated validity. Not sur-prisingly, the reaction of the profession to this volume is a big yawn.
Moreover, the pioneers of personality measurement—J. P. Guilford, Raymond Cattell, and Hans Eysenck—regarded correlations between test scores andreal-world criteria as “peripheral validity”; real validity is the degree to which fac-tor structures replicate across samples. Ironically, the empirical tradition fromMinnesota, which focuses on validity, is typically derided as “dust bowl empiri-cism.” The “pioneers” were interested in replicating factor structures, whereas theempirical tradition is interested in predicting outcomes. The “pioneers” had an ac-ademic agenda that is of little use to consumers of assessment services; the empiri-cal tradition has an applied agenda that interests consumers but not academics.
The last problem concerns the quality of the research in personality psychology.
Like the ability to play the piano, the ability to do research is normally distributed.
A few people, like my wife, have a real talent for research, but most people don’t.
The popularity of meta-analysis has exacerbated the problem because it allowspeople to do research who have little talent for it. Many of the meta-analyses thatevaluate the validity of personality have significant limitations. For example, re-searchers often include in the same analysis measures that are not commensurable.
Thus, they combine measures of normal personality with measures ofpsychopathology and values and interests. In addition, they include well validatedmeasures (e.g., the California Psychological Inventory , or CPI) with poorly vali-dated measures (e.g., the Myers–Briggs and the self-monitoring scale).
Moreover, high scores on the Agreeableness scale of the NEO Personality In- ventory indicate people who try not to give offense, whereas high scores on theLikeability scale of the Hogan Personality Inventory (HPI) indicate people who areactively charming. These scales predict different things and don’t belong in thesame analysis—but they are routinely combined. Furthermore, researchers oftenfail to align predictors with criteria; this results in using measures of conscien-tiousness to predict service orientation, or measures of extraversion to predicttraining performance. The resulting correlations are low and critics then use themto indict personality research rather than the personality researchers. And thenmany researchers ignore the problem of bidirectionality—sometimes measures ofconscientiousness are positively correlated with outcomes and sometimes nega-tively, but for sound theoretical reasons in both cases. Imagine that conscientious-ness is negatively correlated with one kind of performance (rated creativity), butpositively correlated with another kind of performance (rated compliance withrules). This suggests that conscientiousness is a robust predictor of performance.
However, if you simply add the two sets of correlations together, they cancel eachother, and lead to the conclusion that personality is a weak predictor of perfor-mance. Finally, the best known meta-analyses regarding the validity of cognitiveability measures used one test, the General Aptitude Test Battery, because thisavoids the problems of classification and measurement equivalence. In the sameway, meta-analyses of personality research should only use one inventory, ratherthan combining a bunch of them, all of which have different measurement goals.
We judge piano playing by the performance of the best players, not by the per- formance of the average players. Similarly, we should judge personality researchby the performance of the best researchers, not by the performance of the averageresearchers. All researchers are not created equal.
My argument here is frankly ad hominem—many of the critics of personality psy-chology are behaviorist ideologues and this is the source of their hostility to thefield. Most behaviorists will no more be persuaded by data supporting the validityof personality measurement than creationists will be persuaded by data supportingevolutionary theory. As Nils Bohr, the father of atomic theory, remarked, in sci-ence we never persuade our critics, we have to wait for them to die.
The most important claim of personality psychology is that there are structures inside people (hopes, dreams, fears, and aspirations) that determine their behavior.
This claim is anathema to behaviorists. B. F. Skinner, the king of behaviorism, wasnotoriously insensitive to human feelings (when his brother died, Skinner insistedon helping the county medical examiner with the autopsy, which he found quite in-teresting). Skinner went to his grave denying the truth of evolutionary theory andthe existence of instincts—that is, denying that internal structures influence socialbehavior in important ways. Although the modern cognitive behaviorists have nowdiscovered internal structures, they still don’t know how to spell Darwin—that is,they invented structures with no concern for evolutionary theory. A good bit of theantipersonality sentiment in our profession is sheer ideology, promoted by criticswho won’t be persuaded by data.
Note also that academic psychologists don’t compete in the real marketplace of ideas; rather, they play to captive audiences. They sell antipersonality arguments tostudents and like-minded academics, but people in the business community under-stand that individual differences in attitudes and values affect job performance,and they want to use assessment to make better hiring decisions. The problem isthat business people have trouble getting good advice from academic psychology.
This, in turn, explains the widespread interest in bogus measures of personalitysuch as the Myers–Briggs Type Indicator and Goleman’s Emotional CompetenceInventory.
Everyone has a theory of personality; we can’t go to work without one. The prob-lem is that these theories are informal, implicit, and unspecified. The same is truefor industrial psychologists; although they use personality measures, their theoryof personality is rudimentary. That is, they define personality as traits, and that is amistake because trait theory has significant flaws. The problems can be quicklyoutlined. On the one hand, it confuses description with explanation, and is, there-fore, completely tautological. For example, Mike Tyson is usually described as ag-gressive. Trait theorists want to explain Tyson’s aggressive behavior in terms of atrait for aggression, and that’s just dumb.
Sophisticated trait theorists try to escape from the tautology by arguing that reg- ularities in behavior are caused (and explained) by underlying “neuropsychicstructures.” This is psychological reductionism, an effort to explain phenomena atone level in terms of phenomena at the next lower level of analysis. Thus, biologyshould be reduced to the laws of chemistry, chemistry should be reduced to thelaws of nuclear physics, and overt behavior should be reduced to neuropsychicstructures.
There are two problems with reductionism. On the one hand, the rest of science has simply moved on. For example, the Nobel Prize–winning physicist P. W. An-derson (1972) argued 30 years ago that biologists can study biology without wor-rying about chemistry, that chemists can study chemistry without worrying aboutphysics, and so on. More recently, string theory—the theory of everything—is in-tended to make Einstein’s relativity theory (it’s about gravity) consistent withquantum mechanics (particles that have no gravitational properties). But somephysicists think that relativity and quantum mechanics are separate disciplines thatcan be studied fruitfully on their own. Similarly, we can study occupational perfor-mance without resorting to physiology.
The second problem with reductionism in personality psychology is that, after 70 years, we still haven’t found any underlying neuropsychic structures—anddon’t hold your breath. Obviously people are biological animals, and our actionsreflect our genetic makeup, but that is all we need to say. Neuroscientists can studyneuropsychology, and applied psychologists can study social behavior on its ownterms. The bottom line is that trait theory is not a competent theory of personality.
It is a mistake to confuse the way we use trait words with trait theory. Trait words are indispensable for describing other people. However, other people don’t havetraits; rather, we assign trait terms to them as a way of summarizing recurring themesin their behavior. There is a difference between description and explanation, and traittheorists ignore the distinction. We describe other peoples’ behavior with traitwords, but we explain their behavior in terms of what they are trying to accomplish.
Personality is two things: (a) generalizations about human nature, and (b) ex- plorations of individual differences. What generalizations can we make about hu-man nature? Sociology, anthropology, and evolutionary psychology suggest three.
First, people always live in groups. Second, every group has a status hierarchy.
Third, every group has a religion, which is typically used to justify the status hier-archy and the existing moral and legal systems. This suggests that there are threeoverriding themes in individual lives: (a) efforts to get along with other people (be-cause we live with them); (b) efforts to attain some power, status, and control of re-sources (more is always better); and (c) efforts to make some sense out of our lives(by interpreting them in terms of a quasi-philosophical system).
Personality psychology is also about individual differences. People differ from one another in many, many ways. These three generalizations—that people wantacceptance, status, and meaning—suggest what the most important domains of in-dividual differences might be. The first domain will concern individual differencesin the desire for, and the ability to obtain, social acceptance and support. The sec-ond will concern individual differences in the desire for, and the ability to obtain,status, power, and the control of resources. The third will concern individual differ-ences in the desire for meaning and purpose in life.
I have suggested three important generalizations about human nature. I have suggested three important vectors of individual differences. Together, these point to a measurement agenda for personality psychology. In addition, please note thatleaders are people who excel in their ability to gain acceptance and support, powerand status, and to make meaning.
Finally, it is important to note that there is not one definition of personality, there are two. There is personality from the view of the actor, and personality fromthe view of the observer. Personality from the view of the actor—your view ofyou—is identity. Personality from the view of the observer—our view of you—isreputation. Identity and reputation are different, although somewhat related, con-cepts, and they have different implications for assessment. Self-reports—state-ments about who you think you are—are almost useless as data sources in and ofthemselves. As Freud might say, the you that you know is hardly worth knowing—because you made it all up. In any case, the study of identity is not very advanced,and has yielded few reliable generalizations.
On the other hand, reputation has several attractive features as a data source.
First, we have a well-developed vocabulary for talking about reputations, and thatis the vocabulary of trait words. Trait words are what we use to describe other peo-ple, and our descriptions of others are, in fact, their reputations. Second, we have awell-developed taxonomy of trait words, and it is the so-called Five-factor Model.
Recent research indicates that there are more than five dimensions of normal per-sonality, but the point is that we have a well-defined structure in terms of whichtrait terms can be organized. Third, we can use trait words reliably to characterizeothers’ reputations. And finally, reputations are immensely useful for predictivepurposes. The best predictor of future behavior is past behavior, reputations are asummary of past behavior, therefore reputations are the best information we haveabout future behavior.
The measurement model that we use involves taking the statements that people make about themselves and then determining, in an empirical way, the links be-tween their self-descriptions and their reputations. Take, for example, such state-ments as, “I can get this country moving again” or “I can take this company to thenext level.” Our research indicates that peers and subordinates describe peoplewho say this as egotistical narcissists. The trick, then, is to take what people sayabout themselves and translate it into what other people say about them. Con-versely, it is always risky to take what people say about themselves at face value;the data are quite clear that people are poor judges of how they are seen by others(cf. Harris & Schaubroeck, 1988; Mabe & West, 1982). When a manager tells me,“My staff respects me,” I always say, “Can we take a vote?” Before examining the validity data, it might be useful to consider briefly some-thing that the data do not say. The most frequent complaint about personality mea- sures that we hear is that they can be faked, which means that they are useless forpreemployment selection. The empirical literature on this point is vast and extraor-dinarily tedious. Our view of the faking issue can be summarized in terms of fourpoints. First, some people can improve their scores on well-validated measures ofpersonality if you ask them to. However, there are individual differences in this ca-pability, and many people, when trying to “enhance” their scores, actually “de-grade” them. Second, when we evaluate a person for hiring purposes, we look at aprofile, not a score on a single scale. Very few people know what the profile for aparticular job is, and virtually no one can fake an entire profile. Third, althoughsome people can, in principle, alter their scores, it is not clear that job applicants tryto do this, or that it makes any difference. In perhaps the best single study of thistopic, Smith and Ellington (2002) showed that response distortion has little impacton the construct validity of personality measures used in selection contexts. But fi-nally, what does it mean to fake on a personality measure, or to fake in everydaylife? Does it mean to act in a way that is different from the real you? Recall Freud’sview that you made up the you that you know, so that the real you is very hard, ifnot impossible, to define. Imagine that you are walking down a city street and sud-denly feel an urge to poop. If you don’t poop on the sidewalk, then you are not be-ing yourself. You are faking, and after that, it is a slippery slope—in everyday life itis impossible to say where faking ends and authenticity begins. The entire questionof faking founders on the question of authenticity.
What is it that we know about scores on well-constructed measures of personal- ity? One thing we know is that they are remarkably stable over time. Costa, Herbst,McCrae, and Siegler (2000) studied 40-year-olds over a 6- to 9-year interval andreported a median test–retest correlation of .83 over the five factors of the NEOPersonality Inventory. Costa and McRae (2002) provided a nice review of the sta-bility issue. But what about validity? Please consider Table 1. This table concerns the validity of some common med- ical measures and procedures, and provides an essential point of comparison re-garding the size of correlations in other fields. Now consider Table 2. This contains Coronary bypass surgery and 5 year survival Antihistamines and reduced snot and sneezing Effects of Viagra on headaches and flushing These correlations are uncorrected, observed score sample weighted validity coefficients for the seven most common measures usedto predict occupational performance. These correlations, which vary between .11and .26, provide another frame of reference for evaluating the validity of personal-ity measures. Consider next that Judge, Colbert, and Ilies (2004) reported ameta-analytic and fully corrected correlation of .27 between intelligence and lead-ership. These data suggest that a validity coefficient of .30 is unusual at any timefor any measure.
My major claim is that there have always been good data supporting the validity of personality measures, but that the critics of personality measurement won’t payattention. Here are some data from the 1960s. Gough (1965) reported apoint-biserial correlation of .73 between a delinquency–nondelinquency criterionand scores on the Socialization scale of the CPI in a sample of 10,296 people. In arelated study, Gough reported a cross-validated, point-biserial correlation of .63between a CPI regression equation (Socialization had the largest beta weight) andthe delinquency criterion in an American sample of 2,981 cases, and of .60 in aJapanese sample of 149 cases (Gough, DeVos, & Muzushima, 1968). In the beststudy of creativity ever published, Hall and MacKinnon (1969) reported across-validated multiple r of .61 between a three (personality) variable regressionequation and a solid criterion of real-world creativity. Also in the 1960s, I put to-gether a large sample of college students from two campuses in the northeast anddeveloped an index of marijuana use ranging from frequent use to principlednonuse. A CPI-based discriminant function correctly classified 81% of all the us-ers and 81% of all the nonusers (cf. Hogan, Mankin, Conway, & Fox, 1970). Thesestudies only bear indirectly on occupational performance, but they indicate thekind of correlations that one can expect from competent personality research.
More recently, Judge, Bono, Ilies, and Gerhardt (2002) conducted a careful meta-analytic study of the links between personality (defined in terms of theFive-factor Model) and leadership (defined in terms of emergence and effective- ness). They reported the following estimated corrected correlations: (a) Neuro-ticism, –24 (N = 8,025); (b) Extraversion, .31 (N = 11,705); (c) Openness, .24 (N =7,221); (d) Agreeableness, .08 (N = 9,801); and (e) Conscientiousness, .28 (N =7510); with a multiple R, using the five dimensions of .48 (see Table 3).
In the best meta-analytic study of personality and job performance ever pub- lished, Hogan and Holland (2003) confined their investigation to one inventory(the HPI; Hogan & Hogan, 1995)—rather than trying to combine scales across in-ventories—and they carefully aligned predictors with the relevant criteria. That is,they did not try to predict training performance using the HPI Adjustment scale, orsales performance using the HPI Prudence scale. They reported the following esti-mated true validities: (a) Emotional Stability, .43 (N = 2,573); (b) Extraversion andAmbition, .34 (N = 3,698); (c) Agreeableness, .36 (N = 2,500); (d) Conscientious-ness, .36 (N = 3,379); and (e) Intellect and Openness, .34 (N = 1,190; see Table 4).
Big Five Personality Dimensions and Leadership k = number of correlations; p = corrected correlation.
That the article was published at all is a small miracle because the reviewers triedvaliantly to suppress it.
The first assessment centers in the United States contained measures of personalityother than simulations, and were used to study the links between personality and oc-cupational performance (cf. Bray, 1982; MacKinnon, 1978; Murray, 1938; OSS As-sessment Staff, 1948). Standard practice has been to include the personality data,along with the data from the simulations, to create a final score which represents acomposite of the judgment of the assessment center raters. Assessment center rat-ings are valid predictors of occupational performance (Gaugler, Rosenthal, Thorn-ton, & Bentson, 1987). By definition, the personality measurement data will be cor-related with the overall assessment center score. The question becomes, to whatextent do personality measures predict overall ratings from assessment centers? Collins et al. (2003) found 524 articles containing correlations between measures of personality and cognitive ability and performance in assessment centers. Fromthis set, they retained those studies in which the personality data could be coded interms of the Five-factor Model; this resulted in a database of 65 correlations in a totalsample size of 9,738. After doing all the relevant corrections, they reported the fol-lowing average construct level correlations with Overall Assessment Center Rat-ings: (a) Cognitive Ability, .67; (b) Extraversion, .50; (c) Emotional Stability, .35;(d) Openness, .25; and (e) Agreeableness, .17. This leads to a multiple R of .84, sug-gesting that most of the valid variance in Overall Assessment Center Ratings can becaptured with good measures of cognitive ability and normal personality.
What can we conclude regarding the use of personality measures to predict oc- cupational performance? We can conclude four things. First, when the research isdone correctly, the correlations between the standard dimensions of normal per-sonality and job performance criteria that are relevant to that dimension are reli-ably above .30, and multiple correlations approach .50. Second, personality pre-dicts occupational performance almost as well as measures of cognitive ability.
Third, unlike cognitive ability measures, personality measures do not discrimi-nate: Blacks get the same scores as Whites, women get the same scores as men, andthose over 40 get somewhat higher scores than those under 40 (reflecting theirgreater maturity). Finally, we have a solid conceptual account for these findings.
Well-constructed measures of personality are designed to predict reputation; per-formance appraisal at work is all about evaluating reputation. The bottom line is,personality measures work pretty well, especially when compared with all theother measures. When listening to the critics, I find myself asking, “If you don’tlike personality measures, what are the alternatives?” Anderson, P. W. (1972). More is different. Science, 177, 393–396.
Bray, D. W. (1982). The assessment center and the study of lives. American Psychologist, 37, 180–189.
Collins, J. M., Schmidt, F. L., Sanchez-Ku, M., Thomas, L., McDaniel, M. A., & Le, H. (2003). Can ba- sic individual differences shed light on the construct meaning of assessment center evaluations? In-ternational Journal of Selection and Assessment, 11, 17–29.
Costa, P. T., Jr., Herbst, J. H., McCrae, R. R., & Siegler, I. C. (2000). Personality at midlife. Assessment, Costa, P. T., Jr., & McCrae, R. R. (2002). Looking backward: Changes in the mean levels of personality traits from 80 to 12. In D. Cervone & W. Mischel (Eds.), Advances in personality science (pp.
219–237). New York: Guilford.
Gaugler, B. B., Rosenthal, D. B., Thornton, G. C., & Bentson, C. (1987). Meta-analysis of assessment center validity. Journal of Applied Psychology, 72, 493–511.
Gough, H. G. (1965). Conceptual analysis of psychological test scores and other diagnostic variables.
Journal of Abnormal Psychology, 70, 294–302.
Gough, H. G., DeVos, G., & Muzushima, K. (1968). Japanese validation of the CPI social maturity in- dex. Psychological Reports, 22, 143–146.
Hall, W. B., & MacKinnon, D. W. (1969). Personality inventories as predictors of creativity among ar- chitects. Journal of Applied Psychology, 53, 322–326.
Harris, M. M., & Schaubroeck, J. (1988). A meta-analysis of self-supervisor, self-peer, and peer-super- visor ratings. Personnel Psychology, 41, 43–61.
Hogan, J., & Holland, B. (2003). Using theory to evaluate personality and job performance relations.
Journal of Applied Psychology, 88, 100–112.
Hogan, R., & Hogan, J. (1995). Hogan personality inventory manual. Tulsa, OK: Hogan Assessment Hogan, R., Mankin, D., Conway, J., & Fox, S. (1970). Personality correlates of undergraduate mari- juana use. Journal of Consulting and Clinical Psychology, 35, 58–63.
Judge, T. A., Bono, J. E., Ilies, R., & Gerhardt, M. W. (2002). Personality and leadership: A qualitative and quantitative review. Journal of Applied Psychology, 87, 765–780.
Judge, T. A., Colbert, A. E., & Ilies, R. (2004). Intelligence and leadership: A quantitative review and test of theoretical propositions. Journal of Applied Psychology, 89, 542–555.
Mabe, P. A., & West, S. G. (1982). Validity of self-evaluation of ability: A review and meta-analysis.
Journal of Applied Psychology, 67, 280–296.
MacKinnon, D. W. (1978). In search of human effectiveness. Buffalo, NY: Creative Education Founda- Mount, M. K., & Barrick, M. R. (1995). The big five personality dimensions: Implications for research and practice in human resource management. Research in Personnel and Human Resource Manage-ment, 13, 153–200.
Murray, H. A. (1938). Explorations in personality. New York: Oxford University Press.
OSS Assessment Staff. (1948). Assessment of men. New York: Rinehart.
Paul, A. M. (2004). The cult of personality. New York: Free Press.
Schmitt, N. (2004). Beyond the Big Five: Increases in understanding and practical utility. Human Per- Smith, D. B., & Ellington, J. E. (2002). Substance versus style: A new look at social desirability in moti- vating contexts. Journal of Applied Psychology, 87, 211–219.

Source: http://www.peterberry.com.au/files//hogan_research_articles/in_defense_of_personality_measurement.pdf