Abstracts
Résumé
Bien qu’il existe plusieurs méthodes de notation pour assigner des scores aux répondants d’un questionnaire, peu d’études ont comparé les effets que pourraient avoir les méthodes choisies sur les corrélations entre les scores obtenus et d’autres variables. Cette recherche vise à combler ce manque en comparant les coefficients de corrélation entre les scores générés par sept méthodes de notation à partir de données réelles et, à défaut de données réelles accessibles, huit variables générées aléatoirement. Les résultats montrent que les corrélations sont presque identiques et qu’aucune méthode de notation n’a d’effet systématique sur la force des corrélations obtenues. Ce résultat est conforme aux résultats antérieurs et il est recommandé aux chercheurs de privilégier l’utilisation d’une méthode de notation simple et pouvant être utilisée avec des données manquantes.
Mots-clés :
- méthodes de notation,
- scores factoriels,
- trait latent,
- score vrai
Abstract
Even though several scoring methods exist for scoring questionnaires, few studies have compared the potential effects of scoring methods on the correlations between the scores and other variables. This study seeks to fill this void, by comparing the correlation coefficients between the scores obtained with seven scoring methods from real datasets and, in lieu of available real data, eight randomly generated variables. The results show that the correlations are nearly identical and that no scoring method has a systematic effect on the strength of the resulting correlations. This result reinforces previous results and it is thus recommended that researchers favor the use of simple scoring methods able to handle missing data.
Keywords:
- scoring methods,
- factor scores,
- latent trait,
- true score
Resumo
Embora existam vários métodos de pontuação para atribuir pontuações aos respondentes de um questionário, poucos estudos compararam os efeitos que poderiam ter os métodos escolhidos sobre as correlações entre as pontuações obtidas e as outras variáveis. Esta investigação visa preencher esta lacuna através da comparação dos coeficientes de correlação entre as pontuações geradas por sete métodos de pontuação com base em dados reais e, na ausência de dados reais disponíveis, oito variáveis geradas aleatoriamente. Os resultados mostram que as correlações são quase idênticas e nenhum método de pontuação tem um efeito sistemático sobre a força das correlações obtidas. Este resultado é consistente com resultados anteriores e é recomendado que os investigadores privilegiem a utilização de um método de pontuação simples e que pode ser utilizado com dados ausentes.
Palavras chaves:
- métodos de pontuação,
- pontuações fatoriais,
- traço latente,
- verdadeira pontuação
Appendices
Références
- Adedoyin, O. O., Nenty, H. J., & Chilisa, B. (2008). Investigating the invariance of item difficulty parameter estimates based on CTT and IRT. Educational Research and Reviews, 3(3), 83-93. Retrieved from www.academicjournals.org/journal/ERR/article-full-text-pdf/326BFD43220
- Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573. doi: 10.1007/BF02293814
- Barrett, P. (2011). Invoking arbitrary units is not a solution to the problem of quantification in the social sciences. Measurement: Interdisciplinary Research and Perspectives, 9(1), 28-31. doi: 10.1080/15366367.2011.558783
- Beaulieu, C., De Sève, I. et Provost, C. (2015, mai). Anxiété et perception négative de ses capacités : obstacles à la réussite en première session du collégial. Communication présentée au 83e congrès de l’ACFAS, Rimouski, Québec.
- Beck, A. T., Steer, R. A., & Garbin, M. G. (1988). Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clinical Psychology Review, 8(1), 77-100. doi: 10.1016/0272-7358(88)90050-5
- Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, 71(3), 425-440. doi: 10.1007/s11336-006-1447-6
- Chalmers, P. (2016). Bibliothèque mirt (version 1.17.1) [Logiciel R]. https://github.com/philchalmers/mirt
- Chénier, C. (2015, mai). La méthode de notation a-t-elle une influence sur les opérations statistiques ultérieures ? Communication présentée au 83e congrès de l’ACFAS, Rimouski, Québec.
- Cohen, L., Manion, L., & Morrison, K. (Eds.). (2007). Research methods in education (6th ed.). London/New York: Routledge/Falmer.
- Courville, T. G. (2004). An empirical comparison of item response theory and classical test theory item/person statistics. (Unpublished doctoral dissertation). Texas A&M University, Austin, TX.
- DiStefano, C., Zhu, M., & Mindrila, D. (2009). Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research & Evaluation, 14(20), 1-11. Retrieved from http://pareonline.net/pdf/v14n20.pdf
- Dubé, F., Dufour, F., Chénier, C. et Meunier, H. (2016). Sentiment d’efficacité, croyances et attitudes d’enseignants du collégial à l’égard de l’éducation des étudiants ayant des besoins particuliers. Éducation et francophonie, 44(1), 154-172. doi: 10.7202/1036177ar
- Dufour, F., Meunier, H. et Chénier, C. (2014). Quel est le sentiment d’efficacité personnelle d’étudiants en enseignement en adaptation scolaire et sociale dans le cadre du stage d’intégration à la vie professionnelle, dernier stage de la formation initiale ? Dans L. Portelance, S. Martineau et J. Mukamurera (dir.), Développement et persévérance professionnels dans l’enseignement : oui, mais comment ? (p. 75-92). Québec : Presses de l’Université du Québec.
- Dumenci, L., & Achenbach, T. M. (2008). Effects of estimation methods on making trait-level inferences from ordered categorical items for assessing psychopathology. Psychological Assessment, 20(1), 55-62. doi: 10.1037/1040-3590.20.1.55
- Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272-299. Retrieved from www.statpower.net/Content/312/Handout/Fabrigar1999.pdf
- Fan, X. (1998). Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58(3), 357-381. Retrieved from www2.hawaii.edu/~daniel/irtctt.pdf
- Ferrando, P. J., & Chico, E. (2007). The external validity of scores based on the two-parameter logistic model: Some comparisons between IRT and CTT. Psicologica: International Journal of Methodology and Experimental Psychology, 28(2), 237-257. Retrieved from https://eric.ed.gov/?id=EJ802623
- Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt & F. Ostendorf (Eds.), Personality psychology in Europe7. (pp. 7-28). Tilburg, Netherlands: Tilburg University Press.
- Grice, J. W. (2001). Computing and evaluating factor scores. Psychological Methods, 6(4), 430-450. doi: 10.1037/1082-989X.6.4.430
- Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: SAGE Publications.
- Hartig, J., & Höhler, J. (2008). Representation of competencies in multidimensional IRT models with within-item and between-item multidimensionality. Journal of Psychology, 216(2), 89-101. Retrieved from http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&id=2008-08726-005
- Henson, R. K., & Kyle Roberts, J. (2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66(3), 393-416. doi: 10.1177/001316 4405282485
- Hernandez, R. (2009). Comparison of the item discrimination and item difficulty of the Quick-Mental Aptitude Test using CTT and IRT methods. The International Journal of Educational and Psychological Assessment, 1(1), 12-18. Retrieved from http://ecyor.weebly.com/uploads/7/1/1/4/7114954/article2v2.pdf
- Howell, D. C. (2008). Méthodes statistiques en sciences humaines (6e éd.). Bruxelles, Belgique : De Boeck Université.
- Howell, R. D. (2008). Observed variables are indeed more mysterious than commonly supposed. Measurement: Interdisciplinary Research and Perspectives, 6(1-2), 97-101. doi: 10.1080/15366360802121826
- Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley Publishing Company.
- Luo, H. (2011). Generationof non-normal data: A study of Fleishman’s power method. [Working paper]. Retrieved from www.diva-portal.org/smash/get/diva2:407995/FULLTEXT01.pdf
- MacDonald, P., & Paunonen, S. V. (2002). A Monte Carlo comparison of item and person statistics based on item response theory versus classical test theory. Educational and Psychological Measurement, 62(6), 921-943. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1001.262&rep=rep1&type=pdf
- Magno, C. (2009). Demonstrating the difference between classical test theory and item response theory using derived test data. International Journal of Educational and Psychological Assessment, 1(1), 1-11. Retrieved from http://files.eric.ed.gov/fulltext/ED506058.pdf
- Mair, P., Hatzinger, R., Maier, M. J., & Rusch, T. (2015). Bibliothèque eRm (version 0.15-6) [Logiciel R]. http://r-forge.r-project.org/projects/erm/
- Mazza, A., Punzo, A., & McGuire, B. (2015). Bibliothèque KernSmoothIRT (version 6.1) [Logiciel R]. https://CRAN.R-project.org/package=KernSmoothIRT
- Ménard, L., Legault, F., Nault, G., St-Pierre, L., Raîche, G. et Bégin, C. (2011). Projet de recherche sur l’impact des activités formelles de formation et d’encadrement pédagogiques sur les nouveaux enseignants des cégeps et leurs étudiants. Rapport de recherche. Montréal : Université du Québec à Montréal.
- Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105(1), 156-166. doi: 10.1037/0033-2909.105.1.156
- Michell, J. (2002). Stevens’s theory of scales of measurement and its place in modern psychology. Australian Journal of Psychology, 54(2), 99-104. doi: 10.1080/00049 530210001706563
- Millsap, R. E. (2007). Invariance in measurement and prediction revisited. Psychometrika, 72(4), 461-473. doi: 10.1007/s11336-007-9039-7
- R Core Team (2016). R: A language and environment for statistical computing (version 3.3.1). R Foundation for Statistical Computing, Vienna: Austria.
- Reckase, M. D. (2009). Multidimensional item response theory. New York: Springer Science+Business Media.
- Reise, S. P., & Haviland, M. G. (2005). Item response theory and the measurement of clinical change. Journal of Personality Assessment, 84(3), 228-238. doi: 10.1207/s 15327752jpa8403_02
- Revelle, W. (2016). Bibliothèque psych (version 1.6.4) [Logiciel R]. http://personality-project.org/r/psych
- Rotou, O., Headrick, T. C., & Elmore, P. B. (2002). A proposed number correct scoring procedure based on classical true-score theory and multidimensional item response theory. International Journal of Testing, 2(2), 131-141. doi: 10.1207/ S15327574IJT0202_3
- Rupp, A. A., & Zumbo, B. D. (2004). A note on how to quantify and report whether IRT parameter invariance holds: When Pearson correlations are not enough. Educational and Psychological Measurement, 64(4), 588-599. doi: 10.1177/0013 164403261051
- Rupp, A. A., & Zumbo, B. D. (2006). Understanding parameter invariance in unidimensional IRT models. Educational and Psychological Measurement, 66(1), 63-84. doi: 10.1177/0013164404273942
- Santor, D. A., Ramsay, J. O., & Zuroff, D. C. (1994). Nonparametric item analyses of the Beck Depression Inventory: Evaluating gender item bias and response option weights. Psychological Assessment, 6(3), 255-270. doi: 10.1037/1040-3590.6.3.255
- Schaeffer, G. A., Henderson-Montero, D., Julian, M., & Bené, N. H. (2002). A comparison of three scoring methods for tests with selected-response and constructed-response items. Educational Assessment, 8(4), 317-340. doi: 10.1207/S15326977 EA0804_2
- Scheff, T. (2011). The catastrophe of scientism in social/behavioral science. Contemporary Sociology: A Journal of Reviews, 40(3), 264-268. doi: 10.1177/00943061104 04513
- Streiner, D. L. (2010). Measure for measure: New developments in measurement and item response theory. Canadian Journal of Psychiatry, 55(3), 180-186. doi: 10.1177/ 070674371005500310
- Tabachnick, B. G., & Fidell, L. S. (2012). Using multivariate statistics (6th ed.). Upper Saddle River, NJ: Pearson.
- Trendler, G. (2009). Measurement theory, psychology and the revolution that cannot happen. Theory & Psychology, 19(5), 579-599. doi: 10.1177/0959354309341926
- Tschannen-Moran, A., & Woolfolk Hoy, A. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17, 783-805. Retrieved from http://mxtsch.people.wm.edu/Scholarship/TATE_TSECapturingAnElusiveConstruct.pdf
- Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-70. doi: 10.1177/10944 2810031002
- Xu, T., & Stone, C. A. (2011). Using IRT trait estimates versus summated scores in predicting outcomes. Educational and Psychological Measurement, 72(3), 453-468. doi: 10.1177/0013164411419846
- Zaman, A., Kashmiri, A., Mubarak, M., & Ali, A. (2008, November). Students ranking, based on their abilities on objective type test: Comparison of CTT and IRT. Paper presented at the EDU-COM International Conference, Perth, Australia. Retrieved from http://researchonline.jcu.edu.au/7620/1/7620_Jones_2008.pdf
- Zhang, B. (2010). Assessing the accuracy and consistency of language proficiency classification under competing measurement models. Language Testing, 27(1), 119-140. doi: 10.1177/0265532209347363