151 research outputs found
Meta-regression to explain shrinkage and heterogeneity in large-scale replication projects
No full text<p>Code and data to reproduce results from 'Heyard, R. and Held, L. (2024). Meta-regression to explain shrinkage and heterogeneity in large-scale replication projects'. This is a snapshot of the git repository <a href="https://gitlab.uzh.ch/rachel.heyard/heterogeneity_in_replication_projects">https://gitlab.uzh.ch/rachel.heyard/heterogeneity_in_replication_projects</a> commit 9ed801bf from 30 January 2024.</p>
Dynamic report for "Black box 'consensus meeting': Describing the differences between MSCA consensus meetings and Bayesian ranking recommendations"
No full text<p>Code and data to reproduce results from 'Heyard, R., Pina, D. G., Buljan I., Marusic, A. (2024). Black box ‘consensus meeting’: Describing the differences between MSCA consensus meetings and Bayesian ranking recommendations'. This is a snapshot of the git repository https://gitlab.com/heyardr/msca-br-comparison commit d746dd06 from 14 May 2024.</p>
Datastory: Are women submitting fewer grant proposals?
No full textDatastory: Are women submitting fewer grant proposals?
Studies and surveys confirm that during the COVID-19 pandemic, women’s workload at home has increased. Does that mean women researchers are also submitting fewer proposals to the SNSF? Analyses show that, with one exception, their share has remained stable.
English
German
French
Author(s): Simona Isler, Laura Lots, Rachel Heyard, Julius Mattern, Anne Jorstad
DOI: 10.46446/datastory.women-submitting-fewer-grant-proposals / 10.5281/zenodo.4787948
Publication date: 2021-05-0
Replication of null results - Absence of evidence or evidence of absence?
No full textCode to reproduce results from 'Pawel, S., Heyard, R., Micheloud, C., Held, L. (2023). Replication of "null results" - Absence of evidence or evidence of absence?' This is a snapshot of the git repository https://gitlab.uzh.ch/samuel.pawel/rsAbsence commit e7ff4709 from 28 August 2023
SIRRO Preregistration & Data Management Workshop
No full textAs part of the Swiss Open Research Data Grants, the Swiss Reproducibility Network (SwissRN) organized two half-day workshops for researchers in all empirical disciplines and at all levels at SwissRN institutional members in Switzerland: one about preregistration and registered reports (presented by Evie Vergauwe and Caro Hautekiet) and one about data and research management (presented by Eva Furrer and Rachel Heyard). The two half-day workshops were held at four different locations: the University of Zurich and ETH Zurich (May 6th), the University of Bern (May 31st) and the University of Geneva (June 7th).
In the preregistration and registered report workshop, we covered questions such as (1) Why and how to preregister a study?, (2) What is the difference between a study preregistration and a registered report, and (3) How to deal with potential obstacles regarding study preregistration. In the practical part, we discussed situations one can encounter when preregistering a study or submitting a registered report, and how to deal with these situations. Additionally, participants got the opportunity to preregister a simplified example study to get a first, hands-on experience with preregistration.
In the data management workshop, we covered questions such as (1) How to best manage your data and research projects?, (2) What are the FAIR principles?, and (3) How can good meta data and documentation improve your research output? In the practical part, participants got a first taste of version control using Gitlab.
Contact Preregistration & RR workshop:
Caro - [email protected]
Evie - [email protected]
Contact Data Management Workshop
Rachel - [email protected]
Eva - [email protected]
Face-to-face panel meetings versus remote evaluation of fellowship applications: simulation study at the Swiss National Science Foundation.
No full textOBJECTIVES
To trial a simplified, time and cost-saving method for remote evaluation of fellowship applications and compare this with existing panel review processes by analysing concordance between funding decisions, and the use of a lottery-based decision method for proposals of similar quality.
DESIGN
The study involved 134 junior fellowship proposals for postdoctoral research ('Postdoc.Mobility'). The official method used two panel reviewers who independently scored the application, followed by triage and discussion of selected applications in a panel. Very competitive/uncompetitive proposals were directly funded/rejected without discussion. The simplified procedure used the scores of the two panel members, with or without the score of an additional, third expert. Both methods could further use a lottery to decide on applications of similar quality close to the funding threshold. The same funding rate was applied, and the agreement between the two methods analysed.
SETTING
Swiss National Science Foundation (SNSF).
PARTICIPANTS
Postdoc.Mobility panel reviewers and additional expert reviewers.
PRIMARY OUTCOME MEASURE
Per cent agreement between the simplified and official evaluation method with 95% CIs.
RESULTS
The simplified procedure based on three reviews agreed in 80.6% (95% CI: 73.9% to 87.3%) of applicants with the official funding outcome. The agreement was 86.6% (95% CI: 80.6% to 91.8%) when using the two reviews of the panel members. The agreement between the two methods was lower for the group of applications discussed in the panel (64.2% and 73.1%, respectively), and higher for directly funded/rejected applications (range: 96.7%-100%). The lottery was used in 8 (6.0%) of 134 applications (official method), 19 (14.2%) applications (simplified, three reviewers) and 23 (17.2%) applications (simplified, two reviewers). With the simplified procedure, evaluation costs could have been halved and 31 hours of meeting time saved for the two 2019 calls.
CONCLUSION
Agreement between the two methods was high. The simplified procedure could represent a viable evaluation method for the Postdoc.Mobility early career instrument at the SNSF
Meta-science summer school - Bad Soden Salmünster 2025
No full textThis project contains final presentations from the Meta-science Summer School, and will also be used to link future outputs from projects that continue after the summer school. The summer school was held in Bad Soden-Salmuünster, Germany on June 15-20 2025, and was supported by the QUEST Center for Responsible Research, the German Reproducibility Network, Meta-Rep, iRise and the EXCELScIOR ERA Chair team at the University of Coimbra. Funding was generously provided by the Volkswagen Stiftung. Instructors: Tracey Weissgerber, Anita Bandrowski, Malcolm Macleod, Malte Elson, Vladislav Nachev, Ian Hussey, Rachel Heyard. Additional support: Maximillian Frank
Face-to-face panel meetings versus remote evaluation of fellowship applications
No full textThe data shared here has been used in the following paper: https://bmjopen.bmj.com/content/11/5/e047386.full
proposal_id: an anonymise identifier for the proposal
panel_id: an id referencing to the panel in which the proposal was evaluated
referee1: grade given by referee number 1, on a 6-point scale (A, AB, B, BC, C, D)
referee2: grade given by referee number 2, on a 6-point scale (A, AB, B, BC, C, D)
triage: triage group of the proposal (F: funded, R: rejected, D: discussion)
final_decision_ref: final decision using the reference method (F: funded, R: rejected)
random_selection_ref: random selection using the reference method (TRUE/FALSE)
final_decision_test_2r: final decision using the test method with two referees (F: funded, R: rejected)
random_selection_test_2r: random selection using the test method with two referees (TRUE/FALSE)
referee3: : grade given by referee number 3, on a 6-point scale (A, AB, B, BC, C, D)
final_decision_test_3r: final decision using the test method with three referees (F: funded, R: rejected)
random_selection_test_3r: random selection using the test method with three referees (TRUE/FALSE
- …
