Expertise in Science: Cognitivist and Managerialist Approaches
DOI:
https://doi.org/10.15407/scine21.03.053Keywords:
evaluation, expertise, peer review, cognitivist approach, managerialist approach, multidimensional expert culture, multisubjective evaluationAbstract
Introduction. The advancement of scientific knowledge has traditionally been accompanied by rigorous procedures for professional evaluation, ensuring objectivity, logical consistency, evidentiary support, and validity. These procedures serve as fundamental mechanisms for the self-organization and self-regulation of the scientific system. However, as science becomes increasingly integrated into market-driven processes, evaluation procedures have evolved into components of management technologies.
Problem Statement. This shift has undermined the traditional notion of science as a self-organized, self-regulating system. Simultaneously, the proliferation of open science practices and the emergence of new evaluation requirements — driven by the advancement of artificial intelligence technologies — have reshaped the principles of scientific expertise, offering grounds for cautious optimism regarding the future of scientific progress based on self-organization.
Purpose. This study aims to identify emerging trends in scientific expertise, focusing on the expansion of its
functions, methods, and forms.
Materials and Methods. The study has employed comparative analysis, conceptualization and explication of
key concepts, and problem-oriented analysis.
Results. The characteristics of managerialist and cognitivist approaches to evaluation have been clarified. New trends in scientific expertise have been identified, particularly in relation to the spread of open science practices and the integration of artificial intelligence into evaluation processes. It has been substantiated that the shift in assessment priorities from a cognitive to a managerial approach poses risks to the self-organizing capacity of
science. However, open science practices and evolving assessment criteria linked to artificial intelligence technologies have created opportunities to mitigate these risks.
Conclusions. Contemporary trends in scientific evaluation should be grounded in the expansion of expert
competencies and the cultivation of a multidimensional expert culture. This approach would facilitate polylogue,
foster scholarly discussion, ensure the assessment of both actual and potential scientific outcomes, and promote an
appropriate recognition of negative scientific results.
Downloads
References
Merton, R. K. (1973). The sociology of science: Theoretical and empirical investigations. University of Chicago Press.
Ericsson, K. A., Charness, N., Feltovich, P. J., Hoffman, R. R. (2006). The Cambridge Handbook of Expertise and Expert Performance. New York: Cambridge University Press. https://doi.org/10.1017/CBO9780511816796
Zhang, D. C., Wang, Y. (2021). An Empirical Approach to Identifying Subject Matter Experts for the Development of Situational Judgment Tests. Journal of Personnel Psychology, 20(4), 279. https://doi.org//10.1027/1866-5888/a000279
Shanteau, J., Thomas, R. P., Friel, B., Weiss, D. J., Pounds, J. C. (2001). Identifying Expertise Without a Gold Standard: Four Applications. Conference: International Symposium on Aviation Psychology (5—8 March 2001, Columbus, USA). Columbus, OH Volume.
Kurtov, A. I., Polikashin, O. V., Potikhenskyi, A. I., Aleksandrov, V. M. (2017). Expert evaluations. The Delphi method as a management decision-making technology. Collection of scientific works of Kharkiv Air Force University, 1, 118—122 [in Ukrainian].
Kirjavainen, S., Celik, S. (2023). Environmental Considerations in Engineering: Systemic Differences between Experts and Novices. Proceedings of the International Conference on Engineering Design (ICED23) (24—28 July 2023, Bordeaux, France). Cambridge University Press, 1655—1664. https://doi.org//10.1017/pds.2023.166
Denysova, A. (2019). Using the method of expert evaluations in the quality monitoring system of professional education specialists. The image of a modern teacher, 5(188), 39—43. https://doi.org//10.33272/2522-9729-2019-5(188)-39-43 [in Ukrainian].
Ognevyuk, V. O. (Eds.). (2015). Education: training of experts in the field of education. Kyiv [in Ukrainian].
Novosad, V. P., Seliverstov, R. G., Artym, I. I. (Eds.). (2009). Quantitative methods of expert assessment: scientific method. Development. Kyiv [in Ukrainian].
Montgomery, L. E., Lee, M. D. (2021). Expert and novice sensitivity to environmental regularities in predicting NFL games. Judgment and Decision Making, 16(6), 1370—1391. https://doi.org/10.1017/S1930297500008469
Oleksiyenko, R., Donets, A. (2021). The place of expert assessment in management decision-making. Economy and society, 26. https://doi.org/10.32782/2524-0072/2021-26-59 [in Ukrainian].
Pedchenko, N., Strilets, V., Rudenko, N. (2018). Method of Delphi as an innovative tool for managing asymmetric information in financial relationships of potential investors and small business entities. Marketing and Management of Innovations, 3, 68—80. https://doi.org/10.21272/mmi.2018.3-06
Stichweh, R. (2022). Hierarchies and Universal Inclusion in Scientific Communities. In: Peer review in an Era of Evaluation (Еds. Forsberg E., Geschwind L., Levander S., Wermke W.). Palgrave Macmillan, Cham, 37—52. https://doi.org/10.1007/978-3-030-75263-7_2
Hill, S. (2016). Assessing (for) impact: future assessment of the societal impact of research. Palgrave Communications, 2(1), 1—7. https://doi.org/10.1057/palcomms.2016.73
Viana-Lora, A., Nel-lo-Andreu, M. G. (2021). Approaching the Social Impact of Research Through a Literature Review. International Journal of Qualitative Methods, 20, 1—11. https://doi.org/10.1177/16094069211052189
Bellavista, J., Elboj-Saso, C., García Yeste, C., Villarejo-Carballido, B. (2022). Innovative Methodological Approach to Analyze Innovation and Social Impact. International Journal of Qualitative Methods, 21, 1—9. https://doi.org/10.1177/16094069221083373
Almeida, M., Ranisch, R. (2022). Beyond safety: mapping the ethical debate on heritable genome editing interventions. Humanit Soc. Sci. Commun., 9, 139. https://doi.org/10.1057/s41599-022-01147-y
Moher, D., Bouter, L., Kleinert, S., Glasziou, P., Sham, M. H., ..., Dirnagl, U. (2020). The Hong Kong Principles for assessing researchers: Fostering research integrity. PLoS Biol., 18(7), e3000737. https://doi.org/10.1371/journal.pbio.3000737
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., Rafols, I. (2015). The Leiden Manifesto for research metrics. Nature, 520 (22), 429—431. https://doi.org/10.1038/520429a
Hallonsten, O. (2021). Stop evaluating science: A historical-sociological argument. Social Science Information, 60(1), 7—26. https://doi.org/10.1177/0539018421992204
Malitsky, B. А. (2023). Problems of comprehensive assessment of scientific results. Visnik Nacionalnoi Academii Nauk Ukraini, 9, 24—36. https://doi.org/10.15407/visn2023.09.024 [in Ukrainian].
Egorov, I. Yu., Zhukovich, I. A. (2023). Evaluating researcher performance: emerging trends. Science and science of science, 2(120), 42—58. https://doi.org/10.15407/sofs2023.02.042 [in Ukrainian].
Forsberg, E., Geschwind, L., Levander, S., Wermke, W. (2022). Peer Review in Academia. In: Peer review in an Era of Evaluation (Eds. Forsberg, E., Geschwind, L., Levander, S., Wermke, W.). Palgrave Macmillan, Cham, 3—36. https://doi.org/10.1007/978-3-030-75263-7_1
Langfeldt L., Kyvik, S. (2015). Intrinsic tensions and future challenges of peer review. In: RJ Yearbook 2015/2016. Riksbankens Jubileumsfond & Makadam Publishers.
Hansen, H.F. (2022). The Many Faces of Peer Review. In: Peer review in an Era of Evaluation (Eds. Forsberg, E., Geschwind, L., Levander, S., Wermke, W.). Palgrave Macmillan, Cham, 107—126. https://doi.org/10.1007/978-3-030-75263-7_5
Olsen, J. P. (2007).The institutional dynamics of the European university. In: University dynamics and European integration (Eds. P. Maassen, J. P. Olsen). Springer, 25—54. https://doi.org/10.1007/978-1-4020-5971-1_2
Geschwind, L., Edström, K. (2022). Peer Advocacy: Expressions of Loyalty in Peer Review. In: Peer review in an Era of Eva luation (Eds. Forsberg, E., Geschwind, L., Levander, S., Wermke, W.). Palgrave Macmillan, Cham, 203—222. https://doi.org/10.1007/978-3-030-75263-7_9
The Contextualization Deficit: Reframing Trust in Science for Multilateral Policy. (2023). The Centre for Science Futures. Paris, 2023. https://doi.org/10.24948/2023.10
UNESCO Recommendation on Open Science. (2022). UNESCO. URL: https://unesdoc.unesco.org/ark:/48223/ pf0000379949 (Last accessed: 31.07.2024).
Ross-Hellauer T. (2017). What is open peer review? A systematic review [version 2; peer review: 4 approved]. F1000Research, 6, 588. https://doi.org/10.12688/f1000research.11369.2
Statement on peer reviewed publications. (2024). European Science Foundation. URL: coalition-s.org. https://www. coalition-s.org/statement-on-peer-reviewed-publications/ (Last accessed: 31.07.2024).
Matt, C., Hoerndlein, C., Hess, T., Matt, C., Hoerndlein, C.,Hess, T. (2017). Let the crowd be my peers? How researchers assess the prospects of social peer review. Electron Markets, 27, 111—124. https://doi.org/10.1007/s12525-017-0247-4
Eisen, M. B., Akhmanova, A., Behrens, T. E., Harper D. M., Weigel, D., Zaidi, M. (2020). Implementing a “publish, then review” model of publishing. eLife, 9, e64910. https://doi.org/10.7554/eLife.64910
Avissar-Whiting, M., Belliard, F., Bertozzi, S. M., Brand, A., Brown, K.., Dawson, S., …, Williams, M. (2024). Recommendations for accelerating open preprint peer review to improve the culture of science. PLOS Biology, 22(2), e3002502. https://doi.org/10.1371/journal.pbio.3002502
Lee, C. J., Sugimoto, C. R., Zhang, G., Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2—17. https://doi.org/10.1002/asi.22784
Rovetta, A., Bhagavathula, A. S. (2020). COVID-19-Related Web Search Behaviors and Infodemic Attitudes in Italy: Infodemiological Study. JMIR Public Health Surveill, May 5, 6(2), e19374. https://doi.org/10.2196/19374
Zarocostas, J. (2020). How to fight an infodemic. The Lancet, 395, 676. https://doi.org/10.1016/S0140-6736(20)30461-X
Vlasschaert, C., Topf, J. M., Hiremath, S. (2020). Proliferation of Papers and Preprints During the Coronavirus Disease 2019 Pandemic: Progress or Problems With Peer Review? Adv Chronic Kidney Dis, 27(5), 418—426. https://doi.org/10.1053/j.ackd.2020.08.003
Peer review: how we found 15 million hours of lost time. URL: https://www.aje.com/arc/peer-review-process-15-millionhours-lost-time/ (Last accessed: 31.07.2024).
National plan for open science: Decree of the Cabinet of Ministers of Ukraine of 8.10.2022 No. 892-p. Official Bulletin of Ukraine. 2022, No. 83, P. 38. URL: https://zakon.rada.gov.ua/laws/show/892-2022-р#n11 (Last accessed: 31.07.2024).
On the approval of the action plan for the implementation of the Open Government Partnership Initiative in 2023— 2025: Decree of the Cabinet of Ministers of 17.11.2023 No. 1049. Official Bulletin of Ukraine. 2023, No. 105, P. 342. URL: https://zakon.rada.gov.ua/laws/show/1049-2023-р#Text (Last accessed: 31.07.2024).
About the formation of a working group: Order of the Ministry of Education and Science of Ukraine of 24.05.2024 No. 747. URL: https://mon.gov.ua/npa/pro-utvorennia-robochoi-hrupy747 (Last accessed: 31.07.2024).
Brazil, R. (2024). Illuminating ‘the ugly side of science’: fresh incentives for reporting negative results. Nature, 08 May. https://doi.org/10.1038/d41586-024-01389-7
Rosenthal, R. (1979).The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638—641. https://doi.org/10.1037/0033-2909.86.3.638
Kozlov, M. (2024). So you got a null result. Will anyone publish it? Nature, 631, 728—730. https://doi.org/10.1038/d41586-024-02383-9
First International Workshop on Negative Results in Pervasive Computing. (2022). PerFail 2022. URL: perfail-workshop.github.io (Last accessed: 31.07.2024).
Peltonen, E., Mohan, N., Zdankin, P., Shreedhar, T., Nguyen, T., Bayhan, S. (2023). Perspectives on Negative Research Results in Pervasive Computing. IEEE Pervasive Computing, 22(3), 63—72. https://doi.org/10.1109/MPRV.2023.3273718
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Copyright Notice Authors published in the journal “Science and Innovation” agree to the following conditions: Authors retain copyright and grant the journal the right of first publication. Authors may enter into separate, additional contractual agreements for non-exclusive distribution of the version of their work (article) published in the journal “Science and Innovation” (for example, place it in an institutional repository or publish in their book), while confirming its initial publication in the journal “Science and innovation.” Authors are allowed to place their work on the Internet (for example, in institutional repositories or on their website).
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
