Judge response theory? A call to upgrade our psychometrical account of creativity judgments

Archive ouverte : Article de revue

Myszkowski, Nils | Storme, Martin

Edité par HAL CCSD ; American Psychological Association

International audience. The Consensual Assessment Technique (CAT)—more generally, using product creativity judgments—is a central and actively debated method to assess product and individual creativity. Despite a constant interest in strategies to improve its robustness, we argue that most psychometric investigations and scoring strategies for CAT data remain constrained by a flawed psychometrical framework. We first describe how our traditional statistical account of multiple judgments, which largely revolves around Cronbach’s α and sum/average scores, poses conceptual and practical problems—such as misestimating the construct of interest, misestimating reliability and structural validity, underusing latent variable models, and reducing judge characteristics as a source of error—that are largely imputable to the influence of classical test theory. Then, we propose that the item–response theory framework, traditionally used for multi-item situations, be transposed to multiple-judge CAT situations in Judge Response Theory (JRT). After defining JRT, we present its multiple advantages, such as accounting for differences in individual judgment as a psychological process—rather than as random error—giving a more accurate account of the reliability and structural validity of CAT data and allowing the selection of complementary—not redundant—judges. The comparison of models and their availability in statistical packages are notably discussed as further directions.

Consulter en ligne

Suggestions

Du même auteur

Same Test, Better Scores: Boosting the Reliability of Short Online Intelligence Recruitment Tests with Nested Logit Item Response Theory Models | Storme, Martin

Same Test, Better Scores: Boosting the Reliability of Short Online Intellig...

Archive ouverte: Article de revue

Storme, Martin | 2019-09

International audience. Assessing job applicants’ general mental ability online poses psychometric challenges due to the necessity of having brief but accurate tests. Recent research (Myszkowski & Storme, 2018) sugg...

Commentary on Corradi et al.’s (2019) new conception of aesthetic sensitivity: Is the ability conception dead? | Myszkowski, Nils

Commentary on Corradi et al.’s (2019) new conception of aesthetic sensitivi...

Archive ouverte: Article de revue

Myszkowski, Nils | 2020-11

International audience. Corradi et al. (British Journal of Psychology, 2019) argue that their new conception of visual aesthetic sensitivity (as responsiveness to aesthetic features in one’s preferences) presents se...

Are reflective models appropriate for very short scales? Proofs of concept of formative models using the Ten‐Item Personality Inventory | Myszkowski, Nils

Are reflective models appropriate for very short scales? Proofs of concept ...

Archive ouverte: Article de revue

Myszkowski, Nils | 2018-04-06

International audience

Du même sujet

Ethics and cancer: main theme in human and social sciences in a first-degree medical course. Éthique et cancer: fil rouge dans une formation initiale de sciences humaines et sociales en médecine (PACES) | Aiguier, G.

Ethics and cancer: main theme in human and social sciences in a first-degre...

Archive ouverte: Article de revue

Aiguier, G. | 2013-06-15

International audience

Reinventing science and technology entrepreneurship education: The role of human and social capitals | Yami, Saïd

Reinventing science and technology entrepreneurship education: The role of ...

Archive ouverte: Article de revue

Yami, Saïd | 2021-03

International audience

« Conciliation vie privée vie professionnelle » : plusieurs termes pour une même réalité ? Proposition de modélisation à travers l’usage français | Verstaevel, N.

« Conciliation vie privée vie professionnelle » : plusieurs termes pour une...

Archive ouverte: Article de revue

Verstaevel, N. | 2020-12-31

The role of the leverage effect in the price discovery process of credit markets | Zimmermann, Paul

The role of the leverage effect in the price discovery process of credit ma...

Archive ouverte: Article de revue

Zimmermann, Paul | 2021-01-31

Combating climate change and controlling energy demand: introduction to the special section. Lutte contre le changement climatique et maîtrise de la demande d’énergie : introduction au dossier thématique | Aubrée, Loïc

Combating climate change and controlling energy demand: introduction to the...

Archive ouverte: Article de revue

Aubrée, Loïc | 2017-07-28

International audience

New indices to characterize drawing behavior in humans (Homo sapiens) and chimpanzees (Pan troglodytes). New indices to characterize drawing behavior in humans (Homo sapiens) and chimpanzees (Pan troglodytes): An innovative analysis to understand drawing behavior | Martinet, Lison

New indices to characterize drawing behavior in humans (Homo sapiens) and c...

Archive ouverte: Article de revue

Martinet, Lison | 2021

International audience

Chargement des enrichissements...