Progress
For GYA members and alumni only: connect to the group, collaborate and discuss with other members on the GYA-internal Agora platform
here.
Currently, the group engages in five on-going projects:
- Incorporating Intangible Information for Assessment Optimisation
The idea arose from the discussion at the 2017 GYA Annual general Meeting within the working group and was preliminary implemented in 2018. We prepared an anonymous relational database of the applicants and the reviewers with the gender, nationality, and discipline attributes. We also designed a short questionnaire to be filled out during the member selection process 2018 and 2019 by the selection committee members. The questionnaires were collected and preliminary analysis was performed. We did few overall preliminary analyses and planning to perform further analysis using the 2018 and 2019 applications. The ultimate aim is to initiate a new project for identifying novel factors which reflect the implicit personal qualities of a person which are yet vital professionally. Read more
here.
- Analyzing Global Views of Scientific Excellence
Koen Vermeir spearheaded and led the first global study of research evaluation practices since 2016. We need this data-informed approach to be able to propose evidence-based recommendations that can improve current evaluation practices and research culture as a whole. In order to have a comparable data set on a global scale, we started the Analysing of Global Views of Institutional Promotion Criteria, focusing on criteria to become a full professor. The first results and a provisional report of this project was presented at the AGM 2018. By now, we have come out with an analysis from 80 documents we obtained across many countries. More documents will be further analysed. We expect to publish a report in the end of 2019.
- Assessment, Publishing Models, and Open Science
The group is looking into following up on the 2018 report on “
Publishing Models, Assessment, and Open Science“. There are some questions in the problem framework that are worth exploring further.
- Study on Reproducibility and Replicability in Science and Engineering
In the spring of 2018, the working group was invited by the United States National Academies of Sciences, Engineering, and Medicine to provide expert input on the factors that affect reproducibility and replication of data produced from Science and Engineering research. A survey of GYA members provided input for a presentation in May 2018, which provided a set of comments and recommendations on replicability, based on GYA members’ input and on their personal expertise. A report on “Young Scientist Perspectives on Replicability and Reproducibility in Science and Engineering” is forthcoming.
- IAP Research Excellence Study: “Strengthening Research Evaluation: A Global Assessment and Dialogue”
Koen Vermier was nominated to join the InterAcademy Partnership working group on “Strengthening Research Evaluation: A Global Assessment and Dialogue”.
The Scientific Excellence Working Group continues to work together with the InterAcademy Partnership (IAP), ICS, ALLEA, the JRC, and several European Commission Expert Groups to drive change in the perception and measurement of scientific excellence. These collaborations have led to significant results that are impacting policy on a local, regional as well as global level (see the section "outcomes").
Currently, we are working with IAP on a proposal on “Evaluating Research: Assessment and Improvement of Current Practice” and we are collaborating with the JRC on a document on measuring the impact of science advice.