News from the Accelerator- April 2022

Quick Read 

Study updates

    • PSA-002 (Object Orientation): The lead team is working on implementing the extensive feedback from Psychonomic Bulletin and Review into the Stage 2 Registered Report.
    • PSA-003 (Gendered Prejudice): The lead team has written a response letter to the PBR feedback and is in the process of finalizing a new draft of the manuscript. 
    • PSA-004 (Accelerated CREP): The lead team is working on the authorship order and authorship credit tracking. All collaborators were sent the revised manuscript to review by May 8th and an invitation to canvas to fill out the authorship form. If you have not received this invitation, please email Savannah Lewis.
    • PSA-005 (Stereotype Threat): Data collection continues through the summer and fall. As of May 02, the team has collected 797 participants across 32 sites!
    • PSA-006 (Trolley Problem): The manuscript has final acceptance at Nature Human Behaviour. You can now access the full printed manuscript! 
    • PSA-007 (SPAM-L): The admin team is continually working on coding the experiment in lab.js, piloting the experiment, and setting up the system for material translations. The admin team is also pursuing funding opportunities that will allow for the recruitment of additional languages. The project team will be sending out updates in the near future regarding ways to collect data for this project. If you are interested in becoming a collaborator or want to learn more about 007, please email the lead team.
    • PSA-008 (Minimal Groups): The lead team is continuing to finalize their registered report manuscript for submission to Nature Human Behaviour. Onboarding information for those who have filled out the interest survey will be sent out soon; potential collaborators can still express interest in the project (here)
    • PSA-COVID Rapid Bundle (001/003)- 
      • The PSACR-001 lead team has now resubmitted the manuscript to Affective Science.
      • PSACR-003 has been accepted at PNAS. The lead team is currently waiting on paper proofs. PNAS confirmed they will write a letter confirming everyone in the group’s name is a full author. To receive this letter, please email Emma Shumeyko.

Special Call for Studies – Studying Generalizability with Global Samples

The Psychological Science Accelerator (PSA), supported by the John Templeton Foundation (JTF), welcomes study proposals to test the generalizability of phenomena related to JTF strategic priorities with large, global samples.

To be eligible for this call, projects must investigate questions related to one or more of the strategic priorities of JTF: the dynamics of religious change, intellectual humility, religious cognition, the science of character virtue, and health, religion, and spirituality. 

Projects selected through this special call will be advertised to members of the PSA network in collaboration with the proposing authors. Each study will be supported by $40,000 of direct funding for data collection. The proposing author teams will also be collectively supported by 3 full-time scientific staff members (a Research Coordinator, a Postdoctoral Researcher, and a Research Scientist), an additional ⅓ time senior staff member (Dr. Christopher R. Chartier), a ½ time staff member equivalent team of undergraduate research assistants, several members of PSA committees (at their discretion), and members of the network at large (at their discretion).

Selection Process and Timeline

Submission deadline:  July 20, 2022.
Initial feasibility and quality review: July 20- July 27, 2022.
Reviewer selection: July 27- August 3, 2022.
Network rating solicitation: August 3, 2022.
Review submission deadline: August 17, 2022.
Network rating deadline: August 17, 2022
Study selection decision deadline: August 31, 2022. 

Please use this form to submit your proposal.

For pre-submission inquiries, please email Chris Chartier at cchartie@ashland.edu. He is happy to meet with proposing authors and/or answer questions about what is likely to make for a strong submission. To learn more information about submission details and formatting, please visit the official call.

Special Call for Studies – Studying Generalizability with Global Samples

The Psychological Science Accelerator (PSA), supported by the John Templeton Foundation (JTF), welcomes study proposals to test the generalizability of phenomena related to JTF strategic priorities with large, global samples.

The PSA is a distributed network of researchers from the behavioral sciences across the globe. Our more than 1300 collaborators represent 84 countries from all six populated continents. The mission of the PSA is to accelerate the accumulation of reliable and generalizable evidence in psychological science. This call for studies will select projects that fit within the 5 topic areas, broadly construed, that are prioritized by the JTF.

JTF Priority Topics

To be eligible for this call, projects must investigate questions related to one or more of the strategic priorities of JTF. Below is a list of those priorities. Each is followed by some example research questions and topics. These are meant to just be examples; they are not an exhaustive list of the research questions that could be encompassed by each priority.

  • The dynamics of religious change. Why do religions flourish or lose adherents? What kinds of features of religious organizations drive or repel members? Why and how do people switch religious identities?
  • Intellectual humility. Which factors enhance or inhibit intellectual humility? How can we better study the concept of intellectual humility?
  • Religious cognition. What is the nature of religious belief and how can it be better measured? What causes religious or spiritual experiences and what are the effects of those experiences? How do individuals develop and revise their religious beliefs?
  • The science of character virtue. In particular, what helps individuals to develop curiosity and love? What are the consequences of those virtues? How do religions and perceptions of supernatural agents affect virtue and morality development?
  • Health, religion, and spirituality. Are there associations between religious/spiritual beliefs, experiences, practices, and identities, and physical and mental health? If so, are there underlying causal mechanisms?

Proposal Selection

PSA projects are selected using a rigorous review process. Researchers submit detailed proposals to the PSA for consideration. These proposals resemble 5,000 words Stage 1 Registered Reports (e.g., Chambers, 2013), and contain a theoretical introduction, a description of the planned sample and methods, as well as hypotheses and an analysis plan to test those hypotheses. 

The proposals then undergo several rounds of review, overseen by the PSA’s Study Selection Committee (SSC). First, each proposal is screened for feasibility given the PSA’s current capacity and resources. For instance, a proposal that requires dozens of collection sites with fMRI machines is likely to be rejected based on infeasibility. Simultaneously, the SSC screens out submissions deemed low quality. Proposals that pass this screening are then sent out for peer review. Each proposal is reviewed by 5-10 reviewers, consisting of both PSA members and external experts (see Supporting Materials for peer review rating criteria). These reviewers are selected based on their methodological and/or theoretical expertise related to a given proposal. All submissions are also rated quantitatively by the entire PSA network. The (SSC) then aggregates two sources of info, the comments of the reviewers and ratings from the network, before selecting projects for the PSA to pursue.

Funding and Personnel Support for Studies Resulting from this Call

Projects selected through this special call will be advertised to members of the  PSA network in collaboration with the proposing authors. Each study will be supported by $40,000 of direct funding for data collection. The proposing author teams will also be collectively supported by 3 full-time scientific staff members (a Research Coordinator, a Postdoctoral Researcher, and a Research Scientist), an additional ⅓ time senior staff member (Dr. Christopher R. Chartier), a ½ time staff member equivalent team of undergraduate research assistants, several members of PSA committees (at their discretion), and members of the network at large (at their discretion).

Submission Requirements

The following components are required for all submissions:

  • Cover Page, including the title of the study, date of the latest draft, and keywords
  • Abstract of up to 150 words
  • Main body submission text of up to 5,000 words
  • A version of the submission with a cover page included
  • A masked version of the submission without the cover page
  • References
  • Supplementary materials

The following guidelines are intended to assist you in the preparation of your study submission to the PSA. Submissions normally include a description of the key background literature and motivation for the study, hypotheses, study procedures, proposed statistical analysis plan, a statistical power analysis, and pilot data (wherever applicable).

Introduction

A review of the relevant literature that motivates the research question and a full description of the study aims and hypotheses.

Method

A full description of proposed sample characteristics, including criteria for data inclusion and exclusion (e.g., outlier extraction). Procedures for objectively defining exclusion criteria caused by technical errors or for any other reasons must be specified, including details of how and under what conditions data would be replaced.

A description of study procedures in sufficient detail to allow another researcher to repeat the methodology exactly, without requiring further information.

Analysis Plan

Proposed analysis pipeline, including all preprocessing steps, and a precise description of all planned analyses, including appropriate correction for multiple comparisons. Specify all covariates or regressors. Specify analysis decisions that are contingent on the outcome of prior analyses.

Studies involving Neyman-Pearson inference must include a statistical power analysis. Estimated effect sizes should be justified with reference to the existing literature or theory. Because publication bias inflates published estimates of effect size, power analysis should be based on the lowest available or meaningful estimate of the effect size.

In the case of highly uncertain effect sizes, variable sample size and interim data analysis is permissible but with inspection points stated in advance, appropriate Type I error correction for ‘peeking’ employed, and a final stopping rule for data collection outlined.

For studies involving analyses with Bayes factors, the predictions of the theory must be specified so that a Bayes factor can be calculated. Authors should indicate what distribution will be used to represent the predictions of the theory and how its parameters will be specified.

Full descriptions must be provided of any outcome-neutral criteria that must be met for successful testing of the stated hypotheses. Such quality checks might include the absence of floor or ceiling effects in data distributions, positive controls, or other quality checks that are orthogonal to the experimental hypotheses.

Supplemental Materials

Include full questionnaires, stimuli, and materials needed to conduct the study. Pilot data can be included to establish proof of concept, effect size estimations, or feasibility of proposed methods. Simulated data and analysis scripts are recommended to provide clarity about the exclusion criteria and analysis plan.

These guidelines were adapted from https://osf.io/pukzy.

Evaluation Criteria

Two studies will be selected and implemented through this year’s call. The project will also include a second round of submissions, with two additional studies to be selected, in 2023.

In your submission, you should clearly state how your project aligns with the goals of this special call and discuss why your research topic would specifically benefit from global data collection (e.g., theoretical reasons to predict global variation). 

In evaluating submissions for this special call for studies, we will prioritize projects whose designs are most well suited for promoting generalizability. In particular, we will prioritize studies that seek global samples (as opposed to studies that wish to sample from one or a few countries), studies with very high statistical power, and studies that are more strongly grounded in previous research (such as replication and generalization studies). All of these criteria are intended to maximize our chances of producing generalizable insights on phenomena-of-interest.

In addition, we will evaluate proposals for this special call together so that we can ensure variation across the projects. In particular, we will strive to select a package of studies that, collectively, represent a range of JTF priorities, study designs (e.g., experimental vs. correlational), and researcher backgrounds (e.g., research specialty, researcher location, etc.). This way, we will avoid overrepresenting a given topic, type of research, or type of researcher in this initiative. 

Selection Process and Timeline

Submission deadline: July 20, 2022.
Initial feasibility and quality review: July 20- July 27, 2022.
Reviewer selection: July 27-August 3, 2022.
Network rating solicitation: August 3, 2022.
Review submission deadline: August 17, 2022.
Network rating deadline: August 17, 2022
Study selection decision deadline: August 31, 2022.

Possible submission outcomes include “desk rejection” upon initial review, rejection upon full review, provisional acceptance, or an invitation to revise and resubmit for next year’s call.

Following a successful period of needs assessment, preparation, personnel identification, lab recruitment, and pre-registration, provisionally selected studies will commence data collection in early 2023 and end data collection in late 2023.

Please use this form to submit your proposal.

For pre-submission inquiries, please email Chris Chartier at cchartie@ashland.edu. He is happy to meet with proposing authors and/or answer questions about what is likely to make for a strong submission. 

News from the Accelerator- February 2022

Quick Read 

Study updates

  • PSA-002 (Object Orientation): The lead team is working through extensive feedback on the Stage 2 Registered Report submission from Psychonomic Bulletin and Review.
  • PSA-003 (Gendered Prejudice): The lead team has completed initial data analyses and is working on drafting a manuscript describing the results.
  • PSA-004 (Accelerated CREP): The lead team is working on data processing, data analysis, and authorship credit tracking. A group of collaborators were emailed the current version of the manuscript a few weeks ago. 
  • PSA-005 (Stereotype Threat): Data collection continues. As of last week, the team has collected 571 participants across 29 sites! 
  • PSA-006 (Trolley Problem): The manuscript has final acceptance at Nature Human Behaviour. The lead team is waiting to receive the final paper proofs. 
  • PSA-007 (SPAM-L): The team received a favorable revise-and-resubmit on the Stage 1 registered report submitted to Nature Human Behaviour! The project team is now working on (a) addressing reviewer comments, (b) finalizing the experimental materials in lab.js, and (c) establishing a system for material translations. The project team will be sending updates in the near future regarding other ways to contribute. If you are interested in becoming a collaborator or want to learn more, e-mail the lead team.
  • PSA- 008 (Minimal Groups): The lead team thanks everyone who filled out the interest survey and will contact those who responded with onboarding instructions in early April. In the meantime, the lead team is working on a pilot study and a Stage 1 registered report. They aim to submit the manuscript to Nature Human Behaviour by the end of March. If you are interested in becoming a collaborator (and did not fill out the interest survey) or want to learn more about the project, e-mail Kathleen.
  • PSA-COVID Rapid Bundle (001/003)- 
    • The PSACR-001 lead team is working on revising the manuscript at Affective Science. Collaborators have already submitted feedback and approved resubmission, so the paper will be resubmitting once those revisions are complete.
    • PSACR-003 has been accepted at PNAS, but there are still some details being worked out with the journal (particularly regarding payment). Once those details are finalized, collaborators will be notified.

Reminder to apply for the PSA positions at Ashland University.

Christopher Chartier at Ashland University will begin looking at applications for three full-time positions on March 14, 2022. The three positions include a Research Scientist, a Postdoctoral Researcher, and a Research Coordinator that will work with the PSA and Chris on a three-year funded by the John Templeton Foundation. Applicants who submit before on or before March 14 will receive priority when it comes to filling the positions. 

All positions will begin on July 1, 2022 on 1-year contracts through Ashland University with the potential to renew for up to two additional years through the end of the project on June 30, 2025. You can read more about the project here

If you have questions feel free to slack, message or email Chris (cchartie@ashland.edu). 

Links to application and job description: 

Research Scientist
Post-doctoral Researcher 
Research Coordinator 

2022-2025 Vision Plan 

Several members of PSA leadership have drafted a vision plan that reviews the history, guiding principles, and recent accomplishments of the PSA. This document also focuses on 6 priorities the network can focus on over the next three years.

To view the document in its entirety, view this link. If  you would like to submit feedback, please use this form.

Recruiting members for the funding and finance committee

The funding and finance committee is recruiting new members to serve in multiple roles that are vital for sustaining the PSA. The general purpose of the funding and finance committee is to serve as an advisory board for the PSA’s Director and Associate Directors on all matters related to finance and funding. To learn more information, please read this document! 

Call for Proposals: Rare populations!

The PSA is releasing a call for proposed studies with rare populations. To learn more, please read this document!

Community Building and Network Expansion Committee Interns!  

The Community Building and Network Expansion Committee (CBNEC) is a committee of the PSA that is tasked with engaging, expanding and assessing the network. This year, among its many other initiatives, the CBNEC is more purposively and systematically recruiting scholars from developing nations. To help in these initiatives, the CBNEC will be recruiting two (2) interns through the Université Grenoble Alpes.. To learn more, please read this document!

News from the Accelerator- March 2022

Quick Read 

  • On April 29th, Nicholas Coles and Heidi Baumgartner (from ManyBabies) will be hosting a free online panel on authorship challenges in big team science.
  • PSA-005 has officially collected more participants than any other study focused on stereotype threat with African American students.

Study updates

  • PSA-002 (Object Orientation): The lead team is working on implementing the extensive feedback from Psychonomic Bulletin and Review into the Stage 2 Registered Report.
  • PSA-003 (Gendered Prejudice): The lead team has written a response letter to the PBR feedback and is in the process of finalizing a new draft of the manuscript. 
  • PSA-004 (Accelerated CREP): The lead team is working on data processing, data analysis, and authorship credit tracking. Collaborators were added to canvas to fill out the authorship form. If you have not received this invitation, please email Savannah Lewis.
  • PSA-005 (Stereotype Threat): PSA-005 has collected more participants than any other study focused on stereotype threat with African American students, with a total of 681 participants! Congrats to all collaborators on this huge success. Data collection will continue into (Northern Hemisphere) Fall of 2022. 
  • PSA-006 (Trolley Problem): The manuscript has final acceptance at Nature Human Behaviour. The lead team is waiting to receive the final paper proofs. 
  • PSA-007 (SPAM-L): The team is working on revise-and-resubmit from Nature Human Behavior. The admin team is additionally working on coding the experiment in lab.js, piloting the experiment, and setting up the system for material translations. The project team will be sending out updates in the near future regarding ways to contribute to the current project.  If you are interested in becoming a collaborator, or want to learn more about 007 please e-mail the lead team.
  • PSA- 008 (Minimal Groups): The lead team is finalizing their registered report manuscript for submission to Nature Human Behaviour. They have reopened the interest survey to more effectively collect information about potential collaborators and will send out onboarding instructions in the next couple of weeks. 
  • PSA-COVID Rapid Bundle (001/003)- 
    • The PSACR-001 lead team is working on revising the manuscript at Affective Science.
    • PSACR-003 has been accepted at PNAS. Collaborators should receive an update soon from the lead team.

Upcoming Big Team Science Panel 

On April 29th, the director of ManyBabies and the Director of the PSA will be hosting a panel on authorship challenges in big team science. Panelists will include the co-developer of the CRediT taxonomy (Liz Allen), co-developer of the Framework for Open and Reproducible Research Training (Flavio Azevedo), former chair of CASRAI (Simon Kerridge), and metascience researcher at the Meta-Research Innovation Center at Stanford (METRICS) (Robert Thibault). All attendees are welcome and encouraged to bring their own questions and contribute to the conversation.

To register for this (FREE) event, check out his link: 
https://bigteamsciencelab.github.io/events/

PSA Ethics Committee launches a call for new members!

The PSA Ethics Committee is excited to welcome new members and encourages all interested candidates to respond to our call by 25 April (18:00 CET). For more details regarding the mission of the PSA Ethics Committee, the specifics of the call, the qualifications of potential candidates, and the submission procedures please consult the following link. We are expecting you!

Call for Proposals: PSA Rare Populations

The Psychological Science Accelerator is a distributed network of researchers across the behavioral sciences. Great theoretical advances can be made by conducting studies with participants being members of rare populations, but power is necessarily small in single-center investigations (due to the populations being rare). With over 1200 labs from 82 countries on all six populated continents, the PSA has the ability to collect large-sample data from rare populations and advance theory. The PSA is able to contribute to the discovery of generalizable knowledge through a new initiative where researchers may propose a study using a rare population for strong theoretical testing.

What is a rare population? 

For this call, rare populations are defined in a broad manner: these include hard-to-reach samples whether because of logistical, geographic, cultural or other reasons. Examples include people extremely high or low on a given trait; people with a specific diagnosis; members of rare minority groups, trauma survivors and other vulnerable populations, and so on. Rare populations (broadly defined) that can be accessed across multiple countries globally are preferred to rare populations that can only be accessed within a specific country/region. This would ensure the engagement of the full PSA network, increased data gathering capabilities of the study and that the results are generalizable and not dependent on the idiosyncrasies of the local data collection.

What makes for a strong submission?

A major factor of the proposals will be to use a study design where studying the rare population would contribute to a major theoretical advance, regardless of the outcome of the results. In this line, a broad range of methodologies are welcome, not only experimental/quantitative but also including descriptive, exploratory, mixed-methods and qualitative designs. Strong submissions should also justify the need for the use of the PSA network and resources.

Stage 1:

Submit a 2-page proposal with a supplementary document. The proposal should include rationale for the study, the targeted rare population and the study design (including measures, protocols and data analysis).

Importantly, the proposal should come with an additional supplementary document that details the steps to be taken for data gathering labs to access the rare population. As applicable, this document may include protocols to “filter” the general population, partnering with relevant local  institutions, or any such criteria as to demarcate the population of interest and how to access them. This is important so PSA can review the feasibility of the proposal.

Note: Rare Populations can be vulnerable and data collection in some countries can pose ethical or legal risk. Please consider this when preparing your submission and be prepared to submit (separate from the 2-page proposal) discussion of any ethical issues from an international, multi-lab perspective.

Stage 1 screening will be conducted blind to authors’ identity by the PSA Rare Populations Working Group with proposals first screened for depth of theoretical advance as well as feasibility.

Stage 2:

If the 2-page proposal passes the first round, we will be requesting the proposers to write a more detailed proposal with an introduction and methods section in a format that can be submitted as a Registered Report to a major journal [e.g. here].

This longer proposal more fully details the theoretical contributions and methods of the study. Importantly, this should also discuss the specific ethical issues of studying the particular rare population of interest. Potential harms and protocols to mitigate these harms should be discussed. 

Review of Stage 2 Proposals will be conducted blind to the authors’ identity, and will be done by the PSA Rare Populations Working Group and content experts. 

Timeline 

Deadline for the 2-page proposal: April 15, 2022 
Announcement for successful proposals: May 6, 2022 
Deadline for the detailed proposal: June 10, 2022 
Announcement for any selected proposal: July 7, 2022 

Initial 2-page proposal with supplemental steps should be submitted using this Google form here.

For general inquiries, email John Protzko (protzko@gmail.com) with the Subject Line: PSA:RP Question.