Abstract
Quantitative and qualitative studies of scientific performance provide a measure of scientific productivity and represent a stimulus for improving research quality. Whatever the goal (e.g., hiring, firing, promoting or funding), such analyses may inform research agencies on directions for funding policies. In this article, we perform a data-driven assessment of the performance of top Brazilian computer science researchers considering three central dimensions: career length, number of students mentored, and volume of publications and citations. In addition, we analyze the researchers’ publishing strategy, based upon their area of expertise and their focus on venues of different impact. Our findings demonstrate that it is necessary to go beyond counting publications to assess research quality and show the importance of considering the peculiarities of different areas of expertise while carrying out such an assessment.
Similar content being viewed by others
Notes
CNPq website: http://www.cnpq.br
Lattes Plataform: http://lattes.cnpq.br.
Simple H-INdex Estimator: http://shine.icomp.ufam.edu.br.
CAPES Qualis Initiative: http://qualis.capes.gov.br.
Impact factor: http://thomsonreuters.com/journal-citation-reports/.
References
Abramo, G., D’Angelo, C. A., & Di Costa, F. (2011). National research assessment exercises: the effects of changing the rules of the game during the game. Scientometrics, 88(1), 229–238. doi:10.1007/s11192-011-0373-2.
Almeida, E., & Guimarães, J. (2013). Brazil’s growing production of scientific articles—how are we doing with review articles and other qualitative indicators? Scientometrics, 97(2), 287–315. doi:10.1007/s11192-013-0967-y.
Barbosa, S. D. J., & de Souza, C. S. (2011). Are HCI researchers an endangered species in Brazil? ACM Interactions Magazine, 18(3), 69–71. doi:10.1145/1962438.1962454.
Bonaccorsi, A., & Daraio, C. (2003). Age effects in scientific productivity. Scientometrics, 58(1), 49–90. doi:10.1023/A:1025427507552.
Bosquet, C., & Combes, P. P. (2013). Are academics who publish more also more cited? Individual determinants of publication and citation records. Scientometrics, 97(3), 831–857. doi:10.1007/s11192-013-0996-6.
Cerchiello, P., & Giudici, P. (2014). On a statistical h index. Scientometrics, 99(2), 299–312. doi:10.1007/s11192-013-1194-2.
Cota, R. G., Ferreira, A. A., Nascimento, C., Gonçalves, M. A., & Laender, A. H. F. (2010). An unsupervised heuristic-based hierarchical method for name disambiguation in bibliographic citations. Journal of the Association for Information Science and Technology, 61(9), 1853–1870. doi:10.1002/asi.21363.
Delgado-Garcia, J.F., Laender, A.H.F. Jr., W.M. (2014). Analyzing the Coauthorship Networks of Latin American Computer Science Research Groups. In Proceedings of the 9th Latin American Web Congress, Ouro Preto, Brazil, pp 77–81.
Ferreira, A. A., Gonçalves, M. A., & Laender, A. H. F. (2012). A brief survey of automatic methods for author name disambiguation. SIGMOD Record, 41(2), 15–26. doi:10.1145/2350036.2350040.
Glänzel, W., & Schubert, A. (2003). A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics, 56(3), 357–367. doi:10.1023/A:1022378804087.
Glänzel, W., Leta, J., & Thijs, B. (2006). Science in Brazil. Part 1: A macro-level comparative study. Scientometrics, 67(1), 67–86. doi:10.1007/s11192-006-0055-7.
Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16,569–16,572. doi:10.1073/pnas.0507655102.
Ibáñez, A., Larrañga, P., & Bielza, C. (2013). Cluster methods for assessing research performance: exploring Spanish computer science. Scientometrics, 97(3), 571–600. doi:10.1007/s11192-013-0985-9.
Ingwersen, P., Larsen, B. (2014). Influence of a performance indicator on Danish research production and citation impact 2000–2012. Scientometrics, 101(2), 1325–1344. doi:10.1007/s11192-014-1291-x.
Kato, M., & Ando, A. (2013). The relationship between research performance and international collaboration in chemistry. Scientometrics, 97(3), 535–553. doi:10.1007/s11192-013-1011-y.
Kutlar, A., Kabasakal, A., & Ekici, M. (2013). Contributions of Turkish academicians supervising phd dissertations and their universities to economics: An evaluation of the 1990–2011 period. Scientometrics, 97(3), 639–658. doi:10.1007/s11192-013-0973-0.
Laender, A. H. F., de Lucena, C. J. P., Maldonado, J. C., de Souza e Silva, E., & Ziviani, N. (2008). Assessing the research and education quality of the top Brazilian computer science graduate programs. SIGCSE Bulletin, 40(2), 135–145. doi:10.1145/1383602.1383654.
Lamont, M. (2012). Toward a comparative sociology of valuation and evaluation. Annual Review of Sociology, 38(21), 201–221. doi:10.1146/annurev-soc-070308-120022.
Lane, J. (2010). Let’s make science metrics more scientific. Nature, 464(7288), 488–489. doi:10.1038/464488a.
Lee, D., Kang, J., Mitra, P., Giles, C. L., & On, B. W. (2007). Are your citations clean? Communications of the ACM, 50(12), 33–38. doi:10.1145/1323688.1323690.
Leta, J., Glänzel, W., & Thijs, B. (2006). Science in Brazil. Part 2: Sectoral and institutional research profiles. Scientometrics, 67(1), 87–105. doi:10.1007/s11192-006-0051-y.
Lima, H., Silva, T. H. P., Moro, M. M., Santos, R. L. T., Meira, W, Jr, & Laender, A. H. (2013). Aggregating productivity indices for ranking researchers across multiple areas. Proceedings of Joint Conference on Digital Libraries. Indiana, USA: Indianapolis, pp. 97–106. doi:10.1145/2467696.2467715.
Mamtora, J., Wolstenholme, J. K., & Haddow, G. (2014). Environmental sciences research in northern Australia, 2000–2011: A bibliometric analysis within the context of a national research assessment exercise. Scientometrics, 98(1), 265–281. doi:10.1007/s11192-013-1037-1.
Mena-Chalco, J. P., Digiampietri, L. A., Lopes, F. M., & Cesar, R. M. (2014). Brazilian bibliometric coauthorship networks. Journal of the Association for Information Science and Technology, 65(7), 1424–1445. doi:10.1002/asi.23010.
Menezes, G.V., Ziviani, N., Laender, A.H., Almeida, V. (2009). A Geographical Analysis of Knowledge Production in Computer Science. In: Proceedings of International World Wide Web Conference, Madrid, Spain, pp. 1041–1050. doi:10.1145/1526709.1526849.
Miller, J., Coble, K. H., & Lusk, J. L. (2013). Evaluating top faculty researchers and the incentives that motivate them. Scientometrics, 97(3), 519–533. doi:10.1007/s11192-013-0987-7.
Oliveira, E. A., Colosimo, E. A., Martelli, D. R., Quirino, I. G., Oliveira, M. C. L., Lima, L. S., et al. (2012). Comparison of Brazilian researchers in clinical medicine: are criteria for ranking well-adjusted? Scientometrics, 90(2), 429–443. doi:10.1007/s11192-011-0492-9.
Riikonen, P., & Vihinen, M. (2008). National research contributions: A case study on finnish biomedical research. Scientometrics, 77(2), 207–222. doi:10.1007/s11192-007-1962-y.
Sugimoto, C. R., & Cronin, B. (2012). Biobibliometric profiling: An examination of multifaceted approaches to scholarship. Journal of the Association for Information Science and Technology, 63(3), 450–468. doi:10.1002/asi.21695.
Torrisi, B. (2014). A multidimensional approach to academic productivity. Scientometrics, 99(3), 755–783. doi:10.1007/s11192-013-1149-7.
Vanecek, J. (2014). The effect of performance-based research funding on output of r&d results in the Czech Republic. Scientometrics, 98(1), 657–681. doi:10.1007/s11192-013-1061-1.
Veloso, A., Meira, W, Jr, Gonçalves, M., & Zaki, M. (2007). Multi-label lazy associative classification. In: Proceedings of European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases. Warsaw, Poland, pp. 605–612.
Wainer, J., Xavier, E. C., & de Lima, Bezerra F. (2009). Scientific production in computer science: a comparative study of Brazil and other countries. Scientometrics, 81(2), 535–547. doi:10.1007/s11192-008-2156-y.
Wainer, J., Eckmann, M., Goldenstein, S., & Rocha, A. (2013). How productivity and impact differ across computer science subareas. Commun ACM, 56(8), 67–73. doi:10.1145/2492007.2492026.
Acknowledgments
The authors would like to thank Isac Sandin (UFMG) for his invaluable help with processing data from the Lattes Platform, as well as the team behind the SHINE project, notably Prof. Altigran S. da Silva (UFAM). The authors would also like to acknowledge the financial support from InWeb—National Institute of Science and Technology for the Web, and their individual grants from CNPq and FAPEMIG.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lima, H., Silva, T.H.P., Moro, M.M. et al. Assessing the profile of top Brazilian computer science researchers. Scientometrics 103, 879–896 (2015). https://doi.org/10.1007/s11192-015-1569-7
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-015-1569-7