Standardization of sociological tests in the context of measurement problem in sociology
Abstract
The article is devoted to the analysis of a number of problems associated with the use of a specific method of obtaining sociological information - a sociological test. An overview of scientific approaches to understanding the concept of "sociological test" is presented, some related concepts are clarified (in particular, "test", "norm", "normative scale"), procedural aspects related to the development of sociological tests, methods for ensuring their diagnostic capacity are considered. Particular attention is paid to the issues of standardization of sociological tests, since this procedure, firstly, unifies the testing process and ensures objectivity and reproducibility, and, secondly, is an important prerequisite for checking a sociological test for reliability, validity and invariance, which generally determine the quality of this measuring tool. It is noted that, unlike psychological science, where standardization has become an integral part of research practice, sociological questionnaires are often developed without comprehensive verification of the internal structure, which leads to significant variability in the results obtained and complicates the comparison of data from different research centers. In the absence of tools, sociological research risks becoming fragmentary and unsuitable for the comparison procedure, which, in turn, limits the possibility of forming cumulative sociological knowledge, applying the obtained data in interdisciplinary and international comparative studies. These problems are particularly acute in the measurement of multidimensional and context-dependent social constructs (social trust, civic engagement, social status, etc.). The article also outlines a number of methodological guidelines for the practical implementation of the procedure for standardization of sociological tests, in particular, two aspects of standardization and its general procedure, the process and procedure for scale normalization, etc. are characterized. The article also outlines a number of methodological guidelines for the practical implementation of the procedure for standardizing sociological tests; in particular, it describes two aspects of standardization and its general procedure, the process and procedure for normalizing the scale, etc.
Downloads
References
/References
Ніколаєвська А. М. Використання проективних методик в соціологічних дослідженнях. Вісник Харківського державного університету. 1999. № 439. Матеріали III Харківських Міжнародних психологічних читань. Част. 4, 5. С. 303-305.
Паніна Н. Вимірювання в соціології та принципи застосування тестів у масових опитуваннях. Соціологія: теорія, методи, маркетинг. 1998. № 1-2. С. 53-68.
Дембіцький С. Розробка соціологічних тестів: методологія і практики її застосування. Київ: Інститут соціології НАН України, 2019. 304 с.
Дембіцький С. С. Соціологічні тести: сутність і валідизація. Соціологія: теорія, методи, маркетинг. 2016. № 3. С. 140-155.
Дембіцький С. Принципи визначення критеріальних рівнів соціологічного тесту. Український соціум. 2017. № 4. С. 22-34.
Дембіцький С. С. Вимірювання геополітичних орієнтацій в Україні: конструювання соціологічних тестів "RF-geopol" і "EU-geopol". Соціологія: теорія, методи, маркетинг. 2018. № 4. С. 105-134.
Комплексні вимірювальні інструменти в соціологічних дослідженнях: розроблення, адаптація, обґрунтування достовірності / Є. Головаха, С. Дембіцький, Н. Паніна та ін.; за наук. ред. Є. Головахи та С. Дембіцького. Київ: Інститут соціології НАН України, 2022.
Pinch T. "Testing-One, Two, Three... Testing!": Toward a Sociology of Testing. Science, Technology, & Human Values. 1993. Vol. 18(1). Р. 25-41. DOI: https://doi.org/10.1177/016224399301800103
Marres N., Stark D. Put to the test: For a new sociology of testing. British Journal of Sociology. 2020. Vol. 71(3). Р. 423–443. DOI: https://onlinelibrary.wiley.com/doi/full/10.1111/1468-4446.12746
Potthast J. The sociology of conventions and testing. Social theory now, edited by Claudio E. Benzecry, Monika Krause and Isaac Ariail Reed, Chicago: University of Chicago Press, 2017, 337-360. DOI: https://doi.org/10.7208/9780226475318-013
Edwards N. T. The historical and social foundations of standardized testing: In search of a balance between learning and evaluation. Shiken and Shiken Research Journal: JALT Testing and Evaluations SIG Newsletter. 2006. Vol. 10. Р. 8-16.
Friedrich W. N., Olafson E., Faller K. C. Standardized tests and measures. Interviewing children about sexual abuse: Controversies and best practice. Oxford: Oxford University Press, 2007. Р. 207-225.
Fischer R., Milfont T. L. Standardization in psychological research. International Journal of Psychological Research. 2010. Vol. 3(1). Р. 88-96. DOI: https://doi.org/10.4236/psych.2018.911145
Terman L. M. The measurement of intelligence: an explanation of and a complete guide for the use of the Stanford revision and extension of the Binet-Simon intelligence scale. Boston: Houghton Mifflin Company, 1916. URL: https://www.gutenberg.org/files/20662/20662-h/20662-h.htm
Freeman F. N. Mental tests: Their history, principles and applications. Houghton, Mifflin and Company: 1926. https://doi.org/10.1037/10738-000
Spearman C. The Proof and Measurement of Association Between Two Things. Studies in individual differences: The search for intelligence / J. J. Jenkins & D. G. Paterson (Eds.). New York: Appleton-Century-Crofts, 1961. P. 45–58. https://doi.org/10.1037/11491-005
Cronbach L.J. Essentials of Psychological Test. New York: Harper Routledge, 1970.
Havârneanu G. Validity Criteria of a Standardized Test as an Opportunity for Efficient Assessment Created from the Teleological Perspective of Incremental Learning. Science Journal of Education. 2023. Vol. 11 (4). P. 142-149. DOI: https://doi.org/10.11648/j.sjedu.20231104.14
Bas A., et al. Understanding the development, standardization, and validation process of alternative in vitro test methods for regulatory approval from a researcher perspective. Small. 2021. Vol. 17(15). DOI: https://doi.org/10.1002/smll.202006027
Kyriazos T. A., Stalikas A. Applied psychometrics: The steps of scale development and standardization process. Psychology. 2018. Vol 9(11).P. 2531-2560. DOI: https://doi.org/10.4236/psych.2018.911145
Eteng-Uket S. The Development, Validation, and Standardization of a New Tool: The Dyscalculia Test. Numeracy. 2023. Vol.16(2). P. 1. DOI: https://doi.org/10.5038/1936-4660.16.2.1417
Himelfarb I. A primer on standardized testing: History, measurement, classical test theory, item response theory, and equating. Journal of Chiropractic Education. 2019. Vol. 33(2). P. 151-163. DOI: https://doi.org/10.7899/JCE-18-22
Lee S. Advanced Statistical Techniques for Test Standardization: Enhancing Reliability and Validity in Neuropsychological Assessments. NUMBERANALYTICS, 2025. URL: https://www.numberanalytics.com/blog/advanced-statistical-techniques-for-test-standardization#google_vignette
Flake J. K., Fried E. I. Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in methods and practices in psychological science. 2020. Vol. 3(4). P. 456-465. DOI: https://doi.org/10.1177/2515245920952393
Fabian M. Scale norming undermines the use of life satisfaction scale data for welfare analysis. Journal of Happiness Studies. 2022. Vol. 23(4). P. 1509-1541. DOI: https://doi.org/10.1007/s10902-021-00460-8
Leitgöb H., et al. Measurement invariance in the social sciences: Historical development, methodological challenges, state of the art, and future perspectives. Social science research. (2023). Vol. 110. DOI: https://doi.org/10.1016/j.ssresearch.2022.102805
Meuleman B., et al. Why measurement invariance is important in comparative research. A response to Welzel et al. Sociological Methods & Research. 2023. Vol. 52(3). Р. 1401-1419. DOI: https://doi.org/10.1177/00491241221091755
Welzel, Christian, et al. Non-invariance? An overstated problem with misconceived causes. Sociological Methods & Research. 2023. Vol.52 (3). Р. 1368-1400. DOI: https://doi.org/10.1177/0049124121995521
Kusano K., Napier J., Jost J. The mismeasure of culture: When measurement invariance requirements hinder cross-cultural research in psychology. Preprint. (2024). DOI: https://doi.org/10.31234/osf.io/9qe2k
Bommasani R., Liang P. Trustworthy social bias measurement. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society . 2024. Vol. 7. Р. 210-224). DOI: https://doi.org/10.1609/aies.v7i1.31630
Jacobs A. Z., & Wallach H. Measurement and fairness. FAccT '21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 2021. P. 375 – 385. DOI: https://doi.org/10.1145/3442188.3445901
Ponce Gea A. I., Serrano Pastor F. J. The Cultural Element in the Adaptation of a Test: Proposals and Reflections on Internal and External Influences. Education Sciences. 2022. Vol.12(5). Р. 291. DOI: https://doi.org/10.3390/educsci12050291
Council of the International Test Commission. ITC guidelines for translating and adapting tests. International Journal of Testing. 2018. Vol. 18(2). Р. 101-134. https://doi.org/10.1080/15305058.2017.1398166
Oppenheim A. Questionnaire Design, Interviewing and Attitude Measurement. London, New York: Pinter Publishers, 1992. 303 p.
Zumbo, B. D., Gelin, M. N., & Hubley, A. M. (2002). The construction and use of psychological tests and measures. In the Psychology theme of the Encyclopedia of Life Support Systems (EOLSS), Eolss Publishers, Oxford, UK.
Nikolaievska А. M. (1999). ‘Using projective tests in sociological research’. Bulletin of Kharkiv State University. No. 439. Proceedings of the 3rd Kharkiv International Psychological Readings.Part. 4, 5. P. 303-305 [In Ukrainian]
Panina, N.V. (1998). ‘Measurement in sociology and the principles of using tests in mass surveys’ Sociology: theory, methods, marketing, 1-2, 53-68 [In Ukrainian]
Dembitsky, S. (2019). Development of sociological tests: methodology and practices of its application. Kyiv: Institute of Sociology of the NAS of Ukraine. [In Ukrainian]
Dembitskyi, S. (2016). ‘Sociological tests: Essence and validation’. Sociology: Theory, Methods, Marketing, (3), 140–155 [In Ukrainian]
Dembitskyi, S. S. (2017). ‘Principles for determining criterion levels of a sociological test’. Ukrainian Society, (4), 22–34 [In Ukrainian]
Dembitskyi, S. (2018). ‘Measuring geopolitical orientations in Ukraine: Construction of sociological tests «RF-geopol» and «EU-geopol»’. Sociology: Theory, Methods, Marketing, (4), 105–134 [In Ukrainian]
Holovakha, Ye., Dembitskyi, S., Panina, N., et al. (2022). Complex measurement instruments in sociological research: Development, adaptation, and validation (Ye. Holovakha & S. Dembitskyi, Eds.). Kyiv: Institute of Sociology of the National Academy of Sciences of Ukraine [In Ukrainian]
Pinch, T. (1993). ‘«Testing-One, Two, Three... Testing!»: Toward a Sociology of Testing’. Science, Technology, & Human Values, 18(1), 25-41. DOI: https://doi.org/10.1177/016224399301800103
Marres, N., & Stark, D. (2020). ‘Put to the test: For a new sociology of testing’. British Journal of Sociology, 71(3), 423-443. DOI: https://onlinelibrary.wiley.com/doi/full/10.1111/1468-4446.12746
Potthast, J. (2017). The sociology of conventions and testing. Social theory now, edited by Claudio E. Benzecry, Monika Krause and Isaac Ariail Reed, Chicago: University of Chicago Press, 337-360. DOI: https://doi.org/10.7208/9780226475318-013
Edwards, N. T. (2006). ‘The historical and social foundations of standardized testing: In search of a balance between learning and evaluation’. Shiken and Shiken Research Journal: JALT Testing and Evaluations SIG Newsletter, 10, 8-16.
Friedrich, W. N., Olafson, E., & Faller, K. C. (2007). ‘Standardized tests and measures’. Interviewing children about sexual abuse: Controversies and best practice, Oxford University Press, 207-225
Fischer, R., & Milfont, T. L. (2010). ‘Standardization in psychological research’. International Journal of Psychological Research, 3(1), 88-96. DOI: https://doi.org/10.4236/psych.2018.911145
Terman, L. M. (1916). The measurement of intelligence: an explanation of and a complete guide for the use of the Stanford revision and extension of the Binet-Simon intelligence scale. Boston: Houghton Mifflin Company. Available at: https://www.gutenberg.org/files/20662/20662-h/20662-h.htm
Freeman, F. N. (1926). Mental tests: Their history, principles and applications. Houghton, Mifflin and Company. https://doi.org/10.1037/10738-000
Spearman, C. (1961). ‘The Proof and Measurement of Association Between Two Things’. In J. J. Jenkins & D. G. Paterson (Eds.), Studies in individual differences: The search for intelligence, New York: Appleton-Century-Crofts, 45–58. https://doi.org/10.1037/11491-005
Cronbach, L.J. (1970) Essentials of Psychological Test. New York: Harper Routledge.
Himelfarb, I. (2019). ‘A primer on standardized testing: History, measurement, classical test theory, item response theory, and equating’. Journal of Chiropractic Education, 33(2), 151-163. DOI: https://doi.org/10.7899/JCE-18-22
Bas, A., Burns, N., Gulotta, A., Junker, J., Drasler, B., Lehner, R., ... & Rothen‐Rutishauser, B. (2021). ‘Understanding the development, standardization, and validation process of alternative in vitro test methods for regulatory approval from a researcher perspective’. Small, 17(15), DOI: https://doi.org/10.1002/smll.202006027
Kyriazos, T. A., & Stalikas, A. (2018). ‘Applied psychometrics: The steps of scale development and standardization process’. Psychology, 9(11), 2531-2560. DOI: https://doi.org/10.4236/psych.2018.911145
Eteng-Uket, S. (2023). ‘The Development, Validation, and Standardization of a New Tool: The Dyscalculia Test’. Numeracy, 16(2), 1. DOI: https://doi.org/10.5038/1936-4660.16.2.1417
Havârneanu, G. (2023). ‘Validity Criteria of a Standardized Test as an Opportunity for Efficient Assessment Created from the Teleological Perspective of Incremental Learning’. Science Journal of Education, 11(4), 142-149. DOI: https://doi.org/10.11648/j.sjedu.20231104.14
Lee, S. (2025) 'Advanced Statistical Techniques for Test Standardization: Enhancing Reliability and Validity in Neuropsychological Assessments’. NUMBERANALYTICS. Available at: https://www.numberanalytics.com/blog/advanced-statistical-techniques-for-test-standardization#google_vignette
Flake, J. K., & Fried, E. I. (2020). ‘Measurement schmeasurement: Questionable measurement practices and how to avoid them’. Advances in methods and practices in psychological science, 3(4), 456-465. DOI: https://doi.org/10.1177/2515245920952393
Fabian, M. (2022). ‘Scale norming undermines the use of life satisfaction scale data for welfare analysis’. Journal of Happiness Studies, 23(4), 1509-1541. DOI: https://doi.org/10.1007/s10902-021-00460-8
Bommasani, R., & Liang, P. (2024). ‘Trustworthy social bias measurement’. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 7, 210-224. DOI: https://doi.org/10.1609/aies.v7i1.31630
Jacobs, A. Z., & Wallach, H. (2021). ‘Measurement and fairness’. FAccT '21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. P. 375 – 385. DOI: https://doi.org/10.1145/3442188.3445901
Leitgöb, H., Seddig, D., Asparouhov, T., Behr, D., Davidov, E., De Roover, K., ... & van de Schoot, R. (2023). ‘Measurement invariance in the social sciences: Historical development, methodological challenges, state of the art, and future perspectives’. Social science research, 110. DOI: https://doi.org/10.1016/j.ssresearch.2022.102805
Meuleman, B., Żółtak, T., Pokropek, A., Davidov, E., Muthén, B., Oberski, D. L., ... & Schmidt, P. (2023). ‘Why measurement invariance is important in comparative research. A response to Welzel et al.(2021)’. Sociological Methods & Research, 52(3), 1401-1419. DOI: https://doi.org/10.1177/00491241221091755
Welzel, C., Brunkert, L., Kruse, S., & Inglehart, R. F. (2021). ‘Non-invariance? An Overstated Problem With Misconceived Causes’. Sociological Methods & Research, 52(3), 1368-1400. DOI: https://doi.org/10.1177/0049124121995521. (Original work published 2023)
Kusano, K., Napier, J., & Jost, J. (2024). ‘The mismeasure of culture: When measurement invariance requirements hinder cross-cultural research in psychology’. Preprint. DOI: https://doi.org/10.31234/osf.io/9qe2k
Ponce Gea, A. I., & Serrano Pastor, F. J. (2022). ‘The Cultural Element in the Adaptation of a Test: Proposals and Reflections on Internal and External Influences’. Education Sciences, 12(5), 291. DOI: https://doi.org/10.3390/educsci12050291
Council of the International Test Commission. (2018). ‘ITC guidelines for translating and adapting tests’. International Journal of Testing, 18(2), 101-134. DOI: https://doi.org/10.1080/15305058.2017.1398166
Oppenheim, A. (1992). Questionnaire Design, Interviewing and Attitude Measurement. London and New York: Pinter Publishers.
Zumbo, B. D., Gelin, M. N., & Hubley, A. M. (2002). ‘The construction and use of psychological tests and measures’. In the Psychology theme of the Encyclopedia of Life Support Systems (EOLSS), Eolss Publishers, Oxford, UK.
