Abstract:

This study provides the first large-scale examination of the nomological net and incremental validity of assessment center (AC) dimensions identified by. Using two large managerial samples (N = 4,985), investigate the relationships between seven primary AC dimensions and key individual difference variables—cognitive ability and the Big Five personality traits. Findings indicate that AC dimension scores offer meaningful incremental validity beyond traditional psychometric tests. While overall AC ratings showed modest validity (r = .36, mechanically combined composites of AC dimensions produced substantially higher operational validities (unit-weighted r = .44–.50; optimally weighted r = .45–.52), especially when used in conjunction with cognitive and personality assessments. Notably, dimensions such as “problem solving” and “influencing others” showed the strongest associations with cognitive ability and extraversion, respectively, and provided significant incremental gains in predicting job performance (ΔR² = .09–.12). Moreover, single AC dimensions like “problem solving” yielded comparable predictive utility to full composites when paired with personality or cognitive tests. The results underscore the added value of construct-based dimension scoring over holistic AC ratings, which are often diluted by subjective assessor judgments. This research contributes critical evidence supporting the construct validity and applied utility of AC dimension scoring in managerial selection contexts. It also highlights the potential for slimmed-down ACs targeting high-validity dimensions to improve efficiency without sacrificing predictive accuracy.

References:

[1]. Ackerman, P. L., & Heggestad, E. D., 1997, Intelligence, personality, and interests: Evidence for overlapping traits. Psychological Bulletin, 121, 219–245.

[2]. Arthur, W., Day, E. A., McNelly, T. L., & Edens, P. S., 2003, A meta-analysis of the criterion-related validity of assessment center dimensions. Personnel Psychology, 56, 125–154.

[3]. Arthur, W., & Villado, A. J., 2008, The importance of distinguishing between constructs and methods when comparing predictors in personnel selection research and practice. Journal of Applied Psychology, 93, 435–442.

[4]. Bono, J. E., & Judge, T. A., 2004, Personality and transformational and transactional leadership: A meta-analysis. Journal of Applied Psychology, 89, 901–910.

[5]. Carroll, J. B., 1993, Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press.

[6]. Collins, J. M., Schmidt, F. L., Sanchez-Ku, M., Thomas, L., McDaniel, M. A., & Le, H., 2003, Can basic individual differences shed light on the construct meaning of assessment center evaluations? International Journal of Selection and Assessment, 11, 17–29.

[7]. Connelly, B. S., & Ones, D. S., 2007, Combining conscientiousness scales: Can’t get enough of the trait, baby. Poster presented at the Society for Industrial and Organizational Psychology Conference, New York City, NY, April.

[8]. Connelly, B. S., Ones, D. S., Ramesh, A., & Goff, M., 2008, A pragmatic view of dimensions and exercises in assessment center ratings. Industrial and Organizational Psychology, 1, 127–130.

[9]. Dilchert, S., & Ones, D. S., 2009, Assessment center dimensions: Individual differences correlates and meta-analytic incremental validity. International Journal of Selection and Assessment, 17(3), 254–270, https://doi.org/10.1111/j.1468-2389.2009.00468.x

[10]. ePredix., 2001, Global Personality Inventory Technical Manual. Minneapolis, MN: ePredix.

[11]. Feltham, R., 1988, Assessment centre decision making: Judgmental vs. mechanical. Journal of Occupational Psychology, 61, 237–241.

[12]. Gaugler, B. B., Rosenthal, D. B., Thornton, G. C., & Bentson, C., 1987, Meta-analysis of assessment center validity. Journal of Applied Psychology, 72, 493–511.

[13]. Ghiselli, E. E., Campbell, J. P., & Zedeck, S., 1981, Measurement theory for the behavioral sciences. San Francisco: Freeman.

[14]. Goffin, R. D., Rothstein, M. G., & Johnston, N. G., 1996, Personality testing and the assessment center: Incremental validity for managerial selection. Journal of Applied Psychology, 81, 746–756.

[15]. Grove, W. M., & Meehl, P. E., 1996, Comparative efficiency of informal and formal prediction procedures: The clinical-statistical controversy. Psychology, Public Policy, and Law, 2, 293–323.

[16]. Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C., 2000, Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12, 19–30.

[17]. Hardison, C. M., 2005, Construct validity of assessment center overall ratings (Doctoral dissertation). University of Minnesota, Minneapolis, MN.

[18]. Hardison, C. M., & Sackett, P. R., 2004, Assessment center criterion-related validity: A meta-analytic update. Paper presented at the Society for Industrial and Organizational Psychology Conference, Chicago, IL.

[19]. Hermelin, E., Lievens, F., & Robertson, I. T., 2007, The validity of assessment centres for the prediction of supervisory performance ratings: A meta-analysis. International Journal of Selection and Assessment, 15, 405–411.

[20]. Hough, L. M., Ones, D. S., & Viswesvaran, C., 1998, Personality correlates of managerial performance constructs. Poster presented at the Society for Industrial and Organizational Psychology Conference, Dallas, TX, April.

[21]. Hülsheger, U. R., Maier, G. W., & Stumpp, T., 2007, Validity of general mental ability for the prediction of job performance and training success in Germany: A meta-analysis. International Journal of Selection and Assessment, 15, 3–18.

[22]. Hunter, J. E., 1980, Validity generalization for 12,000 jobs. Washington, DC: US Department of Labor.

[23]. Judge, T. A., Bono, J. E., Ilies, R., & Gerhardt, M. W., 2002, Personality and leadership: A qualitative and quantitative review. Journal of Applied Psychology, 87, 765–780.

[24]. Kuncel, N. R., Hezlett, S. A., & Ones, D. S., 2001, A comprehensive meta-analysis of the predictive validity of the GRE: Implications for graduate student selection. Psychological Bulletin, 127, 162–181.

[25]. Lance, C. E., 2008, Why assessment centers do not work the way they are supposed to. Industrial and Organizational Psychology, 1, 84–97.

[26]. Lance, C. E., Newbolt, W. H., Gatewood, R. D., Foster, M. R., French, N. R., & Smith, D. E., 2000, Assessment center exercise factors represent cross-situational specificity, not method bias. Human Performance, 13, 323–353.

[27]. Lievens, F., Dilchert, S., & Ones, D. S., in press, The importance of exercise and dimension factors in assessment centers: Simultaneous examinations of construct- and criterion-related validity. Human Performance.

[28]. Meehl, P. E., 1954, Clinical vs. statistical prediction: A theoretical analysis and a review of the evidence. Minneapolis, MN: University of Minnesota Press.

[29]. Nunnally, J. C., 1978, Psychometric theory (2nd ed.). New York: McGraw-Hill.

[30]. Ones, D. S., 1993, The construct validity of integrity tests (Doctoral dissertation). University of Iowa, Iowa City, IA.

[31]. Ones, D. S., Dilchert, S., Viswesvaran, C., & Judge, T. A., 2007, In support of personality assessment in organizational settings. Personnel Psychology, 60, 995–1027.

[32]. Ones, D. S., Viswesvaran, C., & Reiss, A. D., 1996, Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81, 660–679.

[33]. Ones, D. S., Viswesvaran, C., & Schmidt, F. L., 1993, Comprehensive meta-analysis of integrity test validities. Journal of Applied Psychology, 78, 679–703.

[34]. Raven, J., Raven, J. C., & Court, J. C., 1998, Manual for the Raven’s Progressive Matrices and Vocabulary Scales. Oxford: Oxford Psychologists Press.

[35]. Roberts, B. W., Chernyshenko, O. S., Stark, S. E., & Goldberg, L. R., 2005, The structure of conscientiousness: An empirical investigation. Personnel Psychology, 58, 103–139.

[36]. Schmidt, F. L., & Hunter, J. E., 1998, The validity and utility of selection methods in personnel psychology. Psychological Bulletin, 124, 262–274.

[37]. Schmit, M. J., Kihm, J. A., & Robie, C., 2000, Development of a global measure of personality. Personnel Psychology, 53, 153–193.

[38]. Scholz, G., & Schuler, H., 1993, Das nomologische Netzwerk des Assessment Centers: Eine Meta analyse. Zeitschrift für Arbeits- und Organisations psychologie, 37, 73–85.

[39]. Viswesvaran, C., & Ones, D. S., 2000, Measurement error in Big Five personality assessment. Educational and Psychological Measurement, 60, 224–235.

[40]. Watson, G. B., & Glaser, E. M., 1980, Watson–Glaser Critical Thinking Appraisal Manual. San Antonio, TX: Psychological Corporation.

[41]. Wesman, A. G., 1965, Wesman Personnel Classification Test Manual. New York: Psychological Corporation.