How should we fund the PSA?

Patrick S. Forscher and Hans IJzerman

Note: This post accompanies three surveys, which are embedded as links in the post’s body. For convenience, you can find direct links to them here.

In our previous post, we argued that the PSA has a grand vision and a budget that cannot easily support it. If the PSA is to fulfill its aspirations, the PSA must increase its funding so that it can support an administrative staff.

In this post, we will assume that the PSA wants to fulfill its grand vision. We will therefore explore ways that the PSA could create the funding streams that are necessary to achieve that vision. We will review four options: 

  1. Grant funding
  2. Charitable giving
  3. Fees-for-service
  4. Membership dues

Most of our energy to date has focused on grant funding and charitable giving. A sustainable funding model will require a mix of all the options so that we can balance their strengths and weaknesses. We can think of the PSA’s funding as a sort of investment portfolio where we devote a percentage of effort to each potential income stream based on the PSA’s financial strategy and risk tolerance.

In the remainder of this post, we will evaluate the strengths and weaknesses of all four funding streams. For each funding stream, we will also review the activities that we have already attempted and what else we can do in that category. 

The post is thorough and therefore long. Here are some big-picture takeaways:

  • We have already tried, quite vigorously, to secure grant funding and charitable giving, with modest success
  • Large-dollar grant funding is high-risk, high-reward and likely cannot be counted on to secure the PSA’s financial future; small-dollar grant funding cannot easily support staff
  • Charitable giving mostly yields small amounts of money, though donated research assistants could help provide labor for the PSA
  • Fees-for-service offer some potentially interesting funding streams, including conference organization (continuing the success of PSACON2020) and translation
  • Membership dues arguably fit the structure of the PSA best out of all funding streams, though we should think carefully about how to implement these without compromising our value of diversity and inclusion

We will argue that the PSA should pursue fees-for-service and membership dues, small-dollar grants, and charitable giving. Large-dollar grants should only be pursued in periods when the PSA can tolerate risky funding strategies. Finally, the PSA will need a bookkeeper and Chief Financial Officer to formalize its finances.

Grant-Writing

Grant-writing involves writing a specific proposal to a formal agency. The agency judges the proposal on its merits and its fit with the agency’s mission and, if the proposal is sufficiently meritorious, issues an award. This is the conventional way that scientists try to fund their work. 

We argue that grant writing is high effort, high risk, potentially high yield, but with high volatility. Grant-writing thus makes for bad planning long-term. 

Strengths

  • Grants can be very big. The big granting institutions have a lot of money. For example, one of the grants we wrote in the past three years was for €10 million. This dictum is not always true — many grants are for amounts closer to €1,000 — but when grants pay off big they pay off big.
  • Many scientists have grant-writing expertise. Because grants are the traditional means through which science is funded, grant-writing expertise is easy to find among the scientific community. This makes grant proposals easier to get off the ground than many other funding activities.

Weaknesses

  • Grants usually fund projects, not infrastructures. Funding an infrastructure like the PSA requires an ongoing investment, whereas projects complete within a few years. For this reason, granting agencies view projects as lower-risk funding priorities. Most grant mechanisms are therefore project-focused rather than infrastructure-focused.
  • Grant-writing is high risk. At the US National Institutes for Health, the US’s largest funder, the overall success rate for its most common grant is 23%. This means that, if the PSA relies exclusively on grant funding, we ought to expect 3 out of every 4 grant proposals to not yield any money. It is hard to build a staffing plan for an organization under such risky conditions.
  • Grant-writing is inefficient. Grant-writing requires so much time and energy with such a low probability of paying off that the expected return may not justify the expected time investment. One computational model suggests that these inefficiencies are inherent to any funding scheme structured as a competition between proposals because scientists waste as much or more effort on the proposals as they would gain from the science that the proposals fund.
  • The money is not flexible. The PSA wants to use its money for many things: grants to under-resourced labs, paying staff, funding competitions to make scientific discoveries, and many other things. Grants have strict accounting rules that rule out many or most of these activities.

What have we done already?

  • One proposal to the European Research Council’s Synergy mechanism. This proposal was project-based and very large (€10 million). Some of the money would have gone directly to the PSA, but making the PSA funding work within the constraints of ERC accounting rules was challenging. Writing this grant took years of preparation, including one month of very intense effort by five PSA members. The proposal was rejected.
  • Two proposals to the US National Science Foundation. Both were somewhat large (~$200K) and may have been able to support PSA staff. Both were rejected.
  • Two proposals to the John T Templeton Foundation. Both proposals are project-based and somewhat large (~$200K). One proposal was rejected and one is in limbo due to Covid-19.
  • Two proposals to the European Research Council’s Marie Curie mechanism. Both proposals were project-based and would have supported postdocs (~€120K). Although the postdocs would not have worked directly for the PSA, some of the proposed work would have indirectly benefited the PSA. Both proposals were rejected.
  • One proposal to Université Grenoble Alpes. This proposal was designed to fund grant-writing for both the PSA and UGA. This proposal was funded and has supported a postdoc at a cost of ~€120K. Although the postdoc officially works for UGA, around half their time is donated to the PSA.
  • One proposal to the Association for Psychological Science. This proposal was for $10,000 USD. All the money had to go toward participant recruitment for the PSACR project. This proposal was funded.
  • One proposal to Amazon. This proposal was small ($1,352 USD) and strictly dedicated to paying for server hosting for PSACR. This proposal was funded.
  • One proposal to the Society for Personality and Social Psychology. This proposal was small ($1,000 USD) and could only be used to support participant recruitment for PSA005. This proposal was funded.
  • One proposal to the Society for Improvement of Psychological Science. This proposal was small ($900 USD) and devoted to offsetting the costs of applying for nonprofit status. This proposal was funded. 

The above grants translate to around $12,862,252 in funds sought and $163,252 received. This demonstrates the risk and volatility in grant funding.

In addition to the above activities, PSA leadership has submitted a large number of letters of inquiry and less formal communications seeking funding opportunities. These did not reach the stage of formal proposals.

What else can we do?

The PSA has explored most of the available options related to grant funding. Thus far, it has had the most success writing small, focused grants. These can support specific activities, such as participant recruitment, but cannot easily support staff.

Summary

Grant-writing for large grants is a high-risk, high-reward activity. Grant-writing for small grants usually requires investment of resources, but also cannot easily fund PSA staff. Grant-writing is also the activity into which the PSA has spent most of its funding energy. The PSA may want to focus its grant-writing activities into smaller, project-focused grants.

Charitable giving

Charitable giving involves contacting individuals to solicit money and other resources. Charitable giving is similar to grant-writing in that some entity gives money to the PSA of their own free will. The method differs from grant-writing in that the giver lacks a structured organization that announces funding guidelines and priorities. 

We argue that charitable giving is moderate effort, low risk, low to medium yield, and medium volatility. Charitable giving allows for somewhat better planning long-term than does grant-writing.  

Strengths

  • Allows direct outreach to supportive members of the community. The PSA has generated a lot of support and goodwill. Direct outreach allows members of the community to express this goodwill in the form of direct financial support. 
  • Unconstrained by deadlines. Many other financial streams, such as grants, come on tight deadlines. Solicitations of charitable giving can happen at any time.
  • The money is (usually) flexible. Grant funding comes with strict accounting rules. Charitable giving sometimes comes with strings attached, but less often.

Weaknesses

  • Donation amounts are often small. The cost for staff salary runs in the tens of thousands of dollars. Most gifts run in the tens of dollars.
  • The overhead for these gifts is often large. This limitation is linked to the first. In contrast to grants, where a payment from a granting agency only needs to be processed once, in a donation drive, payments need to be processed as many times as you have donors. Sometimes these payment processes need to go through a third party, such as PayPal. This can create substantial overhead in both labor and finances.

What have we done already?

  • Set up a Patreon. At the time of writing, this Patreon collects €200 per month. We have used this money to help fund special projects, such as the PSA001 Secondary Analysis Challenge, data collection for PSA001, PSA006, and PSACR, and other miscellaneous expenses.
  • Funding drives. We have conducted at least four funding drives in the past three years. These are usually structured around funding a specific project or activity, such as PSACR, and are often announced via the PSA blog. However, we also conducted a funding drive during the first PSA conference, PSACON2020. It is difficult to estimate precisely how much money these drives have generated, but the sum is likely less than $1000.
  • Direct outreach. We have also done direct outreach to specific people who we have thought might be willing and able to donate to the PSA. Usually we have relied on personal networks for these direct solicitations. This process is smoother now that the PSA is an official nonprofit and can therefore accept direct tax-deductible donations.
  • Speaking fee donations. A few members of PSA leadership have donated fees they’ve received to speak about multi-site research. A typical speaking fee is around $200. This source of income generates money, but is hard to scale to the point where it would support full-time staff.
  • Received donations of research assistant time. Once special form of charitable giving is staffing time that is donated from an organization to the PSA. This system allowed the CO-RE Lab, for example, to donate the time of one of its interns to the PSA. This donation enabled, among other things, the creation of the first Study Capacity Report. This system works best when the research assistant or intern can be directly embedded within a PSA supervisory structure.

What else can we do?

The PSA’s new status as a pending nonprofit gives it more options that fit under charitable giving. The option to accept donated time from research assistants is especially attractive — especially if the donation can be formalized in an official agreement between the PSA and the donating organization. 

The PSA can also do more direct outreach to members, both through the PSA mailing list and at the time of registration with the member website (an option that bears some similarities to “membership fees”, discussed below).

Summary

Charitable giving is highly flexible and has generated some money for the PSA, though often the specific monetary amounts are small. One attractive option for the PSA now that it is a nonprofit is to accept more donations of time from member lab research assistants. This option works best when the agreement is formalized through an official agreement and when the research assistant can be embedded within a PSA supervisory structure.

Fees-for-service

A fee-for-service venture involves directly selling something, rather as a for-profit company would. Some of the proceeds from sales go toward maintaining the fee-for-service venture; the rest go toward supporting other activities within the organization.

One particularly interesting service the PSA can provide is translation. The PSA has extensive experience translating psychological tasks and measures into multiple languages. Moreover, the PSA has experience implementing the translations in software. Finally, if the PSA can help researchers recruit participants from more countries and cultures, that furthers its mission of ensuring that psychological science is more representative of humanity writ large. We will have more to say on this subject below.

Fees-for-service ventures are difficult to classify in terms of their effort, risk, yield, and volatility; much depends on the specific venture under consideration. Nevertheless, we believe they are underexplored and could usefully complement the PSA’s other efforts.

Strengths

  • Can leverage already-existing strengths of the organization. If an organization must already be skilled at a specific task, developing a business around that skill allows the organization to leverage its existing strengths to generate a sustainable income. For example, the PSA must accomplish many things to conduct a multi-site study, including develop a large, global community of scientists and researchers around a common mission, coordinate agreements across many institutions, develop materials that are appropriate across many cultures, translate those materials into many languages, create and maintain large databases about its members, and disseminate knowledge about multi-site studies. Many of these skills can be leveraged into an income-generating enterprise. 
  • Builds a sustainable income stream. Once established, a fee-for-service income stream can sustain itself over long periods of time, buffering against the risks associated with other income streams (for example, a grant running out after three years).
  • The money is flexible. Because the money that comes from a fee-for-service venture is earned rather than donated by a third party, there is no donor who can attach strings to how the money is used. Thus, this money is very flexible and can be used to support staff salaries and any other activities the PSA desires.

Weaknesses

  • Startup risks. Startups require up-front investments and there is no guarantee that a market will materialize. Thus, any investment made into a fee-for-service venture may, in the worst case, be lost.
  • Mission capture. On the other hand, if a fee-for-service venture is very successful, that venture may capture the mission of the non-profit as a whole. Thus, fee-for-service ventures work best when they are well in line with the primary aims of the non-profit.
  • Conflicts of interest. Some fees-for-service ventures may create conflicts of interest for PSA members who work on them, as these ventures may create incentives that conflict with the PSA’s core mission.

What have we done already?

  • A PSA conference. This year, the PSA hosted its first conference, PSACON2020. The PSA charged a $60 registration fee with a generous waiver policy — the conference was planned on the assumption that ⅔ of the participants would receive waivers. The conference had 62 paying attendees (plus many more who received waivers) and brought in $3,720. This money went towards supporting the salary of the PSA’s junior administrator (and the main organizer of the conference), Savannah Lewis.

What else can we do?

Fees-for-service are a largely untapped opportunity for the PSA. Conducting multi-site studies is difficult and the PSA has developed a broad array of institutional knowledge to complete these studies. Much of this institutional knowledge can be leveraged into marketable services.

The success of PSACON2020 suggests that we could continue this conference in the future. If attendance remains stable or grows, this could become a small but consistent funding stream for the PSA.

A second interesting option is starting a dedicated PSA journal. This option entails some risk, as projects published there cannot be published in journals that are currently very prestigious (such as Nature Human Behaviour); authors on projects therefore take on some personal risk in the decision to instead publish at the PSA journal. It is also unclear whether university libraries would be willing to pay subscription fees for a PSA journal. On the other hand, the success of society journals suggests that once this revenue source is established, it could be a large and reliable one.

One last interesting option is translation. Translation of psychological measures and tasks is difficult, yet the PSA has lots of experience pulling off this task. PSA member Adeyemi Adetula has proposed a PSA-affiliated translation service with a first focus on Africa. This blog contains two surveys, one for people interested in being paid translators and one for people who want measures and tasks translated. If you are interested in this idea as a means of funding the PSA, head over to the blog, read the proposal, and complete one or both of the surveys.

Summary

Fees-for-service involves leveraging the PSA’s strengths to sell a service, generating a new income stream. The PSA has started some limited versions of fee-for-service funding, but this is still a largely untapped opportunity. 

In addition to continuing the success of PSACON, the PSA could explore starting a service to provide translations of psychological tasks and measures. Both these activities are in line with the PSA’s core mission — the conference because it builds community and supports members, translation because it provides the translated measures and tasks necessary to recruit participants whose native language is not English.

Membership dues

Membership dues are a special case of fee-for-service funding, but the model fits the structure of the PSA so well that it is worth discussing in its own section. The membership dues model involves charging some amount of money to gain access to some or all of the benefits of PSA membership. This is the model followed by scientific societies, which often gate grant funding and conference participation in exchange for annual dues of, say, $247 (APA) or $237 (APS).

Membership dues also have some overlap with some forms of charitable giving. This is because one way to ask for a donation is at the time of registration through the PSA member website. The major difference between this sort of solicitation and a membership fee is whether the default membership is free or paid. Membership dues do have one major, somewhat hidden, advantage over a solicitation at the time of member registration: due to grant accounting rules, many PSA members who have their own grants cannot spend this money on donations. On the other hand, grant money can typically be spent on a membership fee. Instituting membership dues may therefore unlock grant money that is currently inaccessible to the PSA.

Some implementations of membership dues may conflict with a core value of the PSA, namely diversity and inclusion. If dues are required of all members, they risk pushing out prospective PSA members with less resources. There may be implementations of membership dues (such as a dues structure that allows for fee waivers) that can help mitigate this risk. 

We believe that membership dues are low effort, moderate risk, medium yield, and low volatility. They therefore allow for better long-term planning than many other funding streams. However, we also believe that the specific kinds of risks that membership dues incur require feedback from PSA members before we rush into any decisions. We have created a survey to help seek this feedback; we lay out more context for why we see such promise in dues below.

Strengths

  • Many people have pots of money that cannot be given as gifts. Under the Grant-writing section, we noted that grant funds are inflexible. One consequence of this lack of flexibility is that PSA boosters who have grant funds are often not allowed to use them in charitable gifts because the expense cannot be justified to the granting agency. Membership dues provide a way for these people to justify giving money to the PSA to granting agencies, thus unlocking a resource for the PSA that would otherwise go untapped.
  • Leverages the PSA’s best resource, its community. The great strength of the PSA is that it has created a large international community (1400 members and counting) that believes in its mission. Membership dues allow the PSA to leverage this strength to better secure its long-term sustainability.
  • Dues are linked to PSA member benefits. The PSA provides many benefits to members, including a connection with a large international community of colleagues and collaborators, exposure to cutting edge methods, and access to high-impact that can be added to one’s CV. Membership dues would emphasize that the PSA is only able to provide these things by maintaining a large infrastructure.
  • Precedent in scientific societies. The dues funding model is the same one used by scientific societies, so dues-granted membership should be a concept that is familiar to PSA members.
  • Wide variety of potential implementations. Dues can be charged on a yearly basis or could be tied to joining a study (the cost of joining would be an administrative fee). The PSA could also consider a variety of waiver policies.
  • The money is flexible. The money is earned, so it could go toward supporting everything from staff salaries to ad hoc projects.
  • Builds a stable income stream. Once a membership dues program is in place, the income stream from the program should be relatively stable year-to-year.

Weaknesses

  • Risk of lowering membership. Gating membership by adding a joining fee could lower PSA membership, draining the PSA of its most valuable resource. Some policies could help manage this risk — for example, the membership dues could be suggested rather than required.
  • Risk of driving away members with fewer resources. The PSA has a particular interest in increasing its membership in countries that provide less support for the sciences, such as those in the African continent. Imposing membership dues could put joining the PSA out of reach for these prospective members. Once again, policy could help manage this risk — for example, the PSA could provide generous waivers, just as it did for PSACON.

What have we done already?

Nothing.

What else can we do?

Membership dues come with some real risks, so we want to carefully investigate this option before we take any hasty action. We also want to ensure that whatever action we take has the broad support of the PSA membership. 

We have prepared a survey to investigate what PSA members think about different models of membership dues. By taking this survey you can help us evaluate the feasibility of using this as an income stream to promote the sustainability of the PSA.

Summary

Membership dues is arguably the funding model that best fits the structure of the PSA. However, some implementations of dues may conflict with the PSA value of diversity and inclusion. Membership dues have great potential as a funding stream, but we should think carefully about whether and how to implement them.

Conclusion

We strongly believe that the PSA has the potential to do enormous good for psychological science, and perhaps the social sciences at large. However, fulfilling this potential will require dedicated funding, and to obtain this funding the PSA may have to look beyond its current focus on grant-writing and charitable giving. Of course, the PSA need not abandon these income streams; it can simply balance these activities with efforts in developing funding streams through, say, fees-for-service ventures and membership dues.

If the PSA does develop a more balanced financial strategy, it will need staff to manage this strategy. This probably means appointing a Chief Financial Officer and a bookkeeper. The Center for Open Science, for example, has staff to manage its financial strategy and its books.

The PSA is at a crossroads. We think it has a real opportunity to grow into a more formal and mature organization that carries the big team science torch for psychology at large.

The financial cost of the PSA’s vision

Patrick S. Forscher and Hans IJzerman

In 2017, Chris Chartier shared a blog post that revealed a grand vision for psychology research: psychologists could build a “CERN for psychology” that does for psychology what particle accelerators have done for physics. This “CERN for psychology” would be an organization that harnessed, organized, and coordinated the joint efforts of psychology labs throughout the world to take on large, nationally diverse big team science projects.

The months after Chris’s blog went live revealed that enough people believed in this vision to start building this “CERN for psychology”. These early efforts would evolve into the Psychological Science Accelerator, a network that, according to our recent study capacity report, now spans 1400 members from 71 countries. In these early months, the PSA also collaboratively developed a set of five guiding principles, namely diversity and inclusion, decentralized authority, transparency, rigor, and openness to criticism, that form a coherent vision for the type of science we want psychology to be. We want to help transform psychology to become more rigorous, transparent, collaborative, and more representative of humanity writ large.

Now, three years after its founding, the PSA stands at a crossroads. This crossroads relates to our broad vision of what the PSA is and should be and the means through which we achieve that broad vision. This post will cover the first issue. As we will describe, we believe our early documents point to a vision of the PSA as active, rather than passive, but that a lack of funding streams constrains our ability to achieve that mission.

Minimal and maximal visions of the PSA

Although the PSA was established to coordinate the activities of research labs, there are a wide range of options as to how this coordination is implemented. The specific implementations anchor two radically different visions of the PSA: a minimal vision and a maximal vision.

Imagine a PSA that is radically different from the one we have now: the PSA as a mailing list

This mailing list contains the contact information of people who are willing, in principle, to participate in team science projects. To use the mailing list, people design a team science study and email a description of the study to the list. People who receive a study invitation through the mailing list and freely reply to the study proposer. The mailing list itself is unregulated, so there is no vetting process for any of the emails people send over it, nor is there any support for the people who send invitations through the mailing list. This vision of the PSA is highly minimal in the sense that the PSA plays very little role in coordinating or implementing team science projects. However, this vision is also very low cost, as mailing lists are cheap to set up and almost free to maintain. In a sense, this minimalist version of the PSA already exists in the form of StudySwap — a useful tool, but not a transformative one.

Now imagine a PSA that is a bit more similar to the one we have now: the PSA as an active implementer of big team science projects.

In this vision, the PSA completes all stages of the team science research process. This means that the PSA takes partial ownership over its projects and participates in project decisions. This includes selecting projects to undertake through a rigorous review process, assisting with the theoretical development of studies, and improving on the design of studies by soliciting multiple rounds of feedback from relevant experts. In this vision, the PSA also actively coordinates and manages study teams to ensure that relevant administrative procedures (such as ethics review) are followed and to ensure that the various stages of the project occur on a reasonable schedule. The PSA also takes an active role in communicating completed projects to the world, perhaps by managing its own journal (the Proceedings of the Psychological Science Accelerator) and through its own dedicated press team. Finally, in this vision, the PSA has a variety of procedures to proactively improve its processes, including novel methods research, team science best practice research, project retrospectives, and exit interviews of PSA staffers who decide to leave the organization. This is the deluxe vision of the PSA, a PSA that is active but that requires lots of money and staff to maintain.

These two visions of the PSA — the minimal and maximal — also anchor an entire universe of in-between visions that are not so extreme. However, what is arguably true is that the vision of the PSA laid out in Chris’s blog post, as well as the one implied the PSA’s five guiding principles, are both much closer to the maximal vision of the PSA than the minimal one.

Money is necessary to implement a maximal vision of the PSA

If we accept that fulfilling the PSA’s mission requires something closer to a maximal vision of the PSA, we need to find ways to build the PSA into this more maximal vision. At a minimum, building and maintaining a maximal vision of the PSA requires people who can do the activities involved in this maximal vision. These people need to be recruited, managed, and retained, otherwise they will work at cross purposes, get into interpersonal conflicts, and burn out. In short, in addition to the people who carry out the PSA’s vision, the PSA needs an administrative structure to help these people carry out their work.

The PSA does, in fact, have a defined administrative structure. We have, for example, defined a set of committees to govern its activities and a set of roles that should be filled for each project. These roles outline an aspiration for the PSA to proactively conduct team science research — further suggesting that the PSA has a maximalist vision for itself.

These roles are many. Our recent evaluation of the PSA’s administrative capacity identifies fully 115 of them. If we assume that each position requires 5 hours per week of work to complete the associated responsibilities, the 115 roles require 29,900 hours per year to staff. Unfortunately, maintaining this level of labor has, at times, been challenging because of our reliance on volunteers who have other daily commitments (such as jobs that pay them). 

We can rely on volunteers to carry this load for a time, but doing so carries some real risks. For example, the heavy load can risk burning out the most active volunteers who take on the most labor and lead to large, costly mistakes if the labor requirements for large projects are not met. This load can also provoke interpersonal conflict if active volunteers feel that their labor is not properly recognized or credited. Finally, there are risks associated with who is able to be a volunteer: in a volunteer model, only those who can afford to will donate their time and efforts to the PSA. This squeezes out the voices and talents of some of the very people the PSA wants to elevate, such as those from Africa, Middle America, and South America. Our 2020-2021 study capacity report estimates that 90% of all people involved in administrative roles are from North America and Western Europe. The majority of these, 63% of all administrative roles, are located in North America.  

Other growing organizations have managed the transition from an exclusively volunteer organization to one that is funded by some sort of income stream. We can learn from their history, which shows that the organizations that became sustainable did so by leveraging their already-existing strengths.

The first step down the path of creating a sustainably funded organization involves acknowledging that many of the positions outlined in PSA policies are best served, not as volunteer positions, but as paid positions. We must also acknowledge that the costs of paying for this labor may be high. If we assume that all 29,900 hours are paid, and paid at even a very low wage ($7.25/hour, or US minimum wage), we still get a labor cost of at least $216,775 per year. This does not consider taxes, vacation, or other overhead. 

Of course, it may not be necessary to pay for all 29,900 hours, either because our staffing estimates are inaccurate, or because our labor becomes more efficient when we switch to a paid model. Yet the mere act of thinking through these considerations requires recognizing that the PSA’s maximalist vision has a financial cost.

What we can be is constrained by our ability to obtain resources

The vision of the PSA outlined in its founding documents is grand. If implemented successfully, this vision could have an impact on psychology that is transformative, creating a science that is more inclusive, more collaborative, more transparent, and more robust.

Yet the PSA cannot realize this vision of itself for free. Currently, the PSA attempts to be a maximalist institution on a minimalist budget. That has worked during the PSA’s early years, but such a model may not be sustainable long-term. If we wish to implement a maximal vision for the PSA, we will need to focus dedicated energy into obtaining the funding needed for this implementation. As we will describe in a follow-up post, this will likely require developing funding streams outside of the traditional grant mechanisms to which scientists are accustomed.


Funding Note: Patrick Forscher is paid via a French National Research Agency “Investissements d’avenir” program grant (ANR-15-IDEX-02) at Université Grenoble Alpes awarded to Hans IJzerman.

How many resources does the PSA possess?

Patrick S. Forscher, Bastien Paris, and Hans IJzerman

How many resources does the PSA possess? This is a question that affects many activities within the PSA — prime among them the annual decision of how many studies the PSA is able to accept. Here are a few other examples:

  • People involved in the selection of studies must decide whether the PSA can feasibly support studies with special requirements, such as a proposal to conduct a multi-site EEG project or a project involving a low-prevalence (and therefore difficult to recruit) population. 
  • People involved in writing PSA-focused grants must be able to accurately describe the size and scale of the network to make their grant arguments and planning concrete. 
  • People involved in managing the PSA’s finances need to know the people and projects that have the highest financial need. 
  • People involved in regional recruitment need to know how many members are currently located in a specific world region and the number of participants those members can muster for a typical PSA study. 

In its first three years, we have had to rely on ad hoc sources to answer questions about PSA resources. Today, with the release of the PSA’s first study capacity report, we now have a source that is more systematic. This blog describes the logic that underlies the report, gives some of its top-level findings, and outlines what we plan to do with the report now that we have it.

How to think about and report on PSA resources

The PSA’s most basic activity is running multi-site studies, and one of the most fundamental resource-dependent decisions PSA leadership must make is how many proposals for these multi-site studies the PSA will accept. Thus, a single multi-site study provides a useful yardstick for measuring and thinking about PSA resources.

The PSA’s newly-ratified resource capacity policy takes just such an approach. It considers PSA resources from the perspective of helping PSA leadership decide how many studies they should accept in a given year. From this perspective, the most basic unit of analysis is the study submission slot, a promise by the PSA to take on a new multi-site study. Study submission slots are limited by at least two types of resources:

  1. Data collection capacity. This is the PSA’s ability to recruit participants for multi-site studies. Data collection capacity is mainly governed by the number of PSA members located in psychology labs throughout the world. However, money can also expand the PSA’s data collection capacity; the PSA has occasionally contracted with panel-provider firms to recruit participants on its behalf.
  2. Administrative capacity. This is the PSA’s ability to perform the administrative tasks required to support multi-site studies. Administrative capacity is mainly governed by the availability of labor, whether that labor be paid or volunteer.

The resource capacity policy also allows for the possibility of study slots that add on special requirements or evaluation criteria. These special submission slots might require, for example, that any studies submitted for consideration to that slot involve EEG equipment. Alternatively, the slots might require that the submitted studies involve investigating the psychological aspects of the Covid-19 pandemic. We will go into more detail about how we think about these special submission slots in a later post. For the time being, we simply note that assessing our resource capacity will allow us to understand the sorts of special submission slots we can accommodate.

The PSA’s data collection and administrative capacities are both in flux. The PSA’s ability to accommodate more specialized types of studies also fluctuates on a yearly basis. Moreover, the PSA is committed to the cultural and national diversity of psychology research — activities that are dependent on its reach in under-resourced countries. Accurate assessment of all these capacities therefore requires ongoing documentation of its members, member characteristics (including country of origin), and its yearly activities. Currently, our documentation happens in a shared Google Drive, Slack, the PSA’s OSF project and the subprojects for each of its studies, and the recently-created PSA member website.

According to policy, these various sources of documentation are consulted in a comprehensive way to form a complete picture of the PSA’s resources. This consultation results in an annual  study capacity report, which can inform decisions and activities involving the PSA’s resources.

Findings from the first study capacity report

The first PSA study capacity report is large and comprehensive. Here are some big-picture findings:

  • The PSA currently has 1,400+ members from 71 countries.
  • Out of seven studies, six are still underway collecting data.
  • Based on our past data collection capacity, we have the ability to recruit a minimum of 20,000 participants over the upcoming scholarly year for new PSA projects.
  • Two out of three PSA members come from North America (24%) and Western Europe (41%). 
  • We do not have sufficient information to accurately estimate the number of administrative hours available for each PSA role.

However, these big-picture findings hide a lot of important detail that may be important for PSA decision-making. For example, here are a few additional tidbits that come out of the report:

  • The number of PSA member registrations almost tripled as a result of the COVID-Rapid project.
  • At the time of the report’s writing, and excluding PSA007, the PSA will need to recruit 30,000 participants to complete its active roster of projects.
  • About 20% of the PSA’s membership have a social psychology focus area.
  • About 90% of people in active PSA administrative roles are located in North America (63%) and Western Europe (21%).

If you’re interested in digging into more of these details, you can find the full report here.

What’s next?

As outlined by policy, the main purpose of the report is to inform decisions about how many studies the PSA can accept in the next wave of study submissions. Thus, an important next step for this report is for the upper-level leadership to use the report to come to a decision about study submission slots.

However, the study capacity report has already catalyzed a number of ongoing conversations about what the PSA is, what it should be in the future, and how the PSA should go about meeting its aspirations for itself. Some of these conversations have resulted in their own dedicated blog posts, which will be posted to the PSA blog in the next few days.

In the meantime, we welcome your thoughts about the PSA’s study capacity and issues related to it. We believe that compiling this report has been a useful exercise precisely because the process of compiling the report has inspired so many useful conversations about the PSA’s direction and goals. This reinforces our commitment to maintaining this useful reporting structure in future years.


Funding Note: The study capacity report was made possible via the work of Bastien Paris; his internship at Université Grenoble Alpes is funded by a grant provided by the Psychological Science Accelerator. Patrick S. Forscher is paid via a French National Research Agency “Investissements d’avenir” program grant (ANR-15-IDEX-02) awarded to Hans IJzerman.

News from the Accelerator- October 2020

Study Capacity Policy up for a Vote

In an effort to more systematically assess our study acceptance capacity, we have drafted a new PSA policy document. All members of the PSA, please read this document, and then vote yes or no, by logging into the member site and finding the voting form under the “your tasks” heading on the main page.

Data Collection Updates

  • PSA 004: We have 2640 participants in over 50 labs. The deadline to sign up is past, but if your lab hasn’t started data collection then we will need you to start by November 15. Data collection ends in December. 
  • PSA 006: 006 is in the final stages of data collection. We are going strong but are not accepting any new labs at this time. Data collection is set to end at the end of 2020.
  • PSACR 001/002/003: Data collection for the PSACR bundle was completed on October 23rd! Considering people who completed at least 90% of questions on a given component of the survey, our estimated Ns are: 44,217 for the general questions; 16,618 for Study 1 (Loss Gain); 20,805 for Study 2 (Cognitive Reappraisal); and 18,594 for Study 3 (Self Determination). We also met our goal for Study 2 of having 35 countries with N >= 200. Anyone can explore Ns by study, language, and different percent completion thresholds at http://formr.psysciacc.org/shiny/shiny_app/.

Publication Updates 

  • PSA 001: The final copy of the paper has been accepted and we are working on responding to the various editorial formatting requests prior to publication! This is our first study to have a final, Stage 2, acceptance.
  • PSA 002/003: We are currently waiting on the journal editor’s comments about the modifications we made earlier. He will give us their decisions and suggestions in a few days, so we will have a lot of editing to do soon!
  • PSA 005: We have received Stage 1 acceptance at Nature Human Behavior! Here is a preprint of the manuscript if y’all want to give it a read!

Training Committee Update

We are trying to get some feedback from you all on what kind of training you think you need in order to be successful in the PSA and the many studies. Here is a google form you can all fill out to provide some feedback on what you think would be helpful. 

Community Building and Network Expansion Committee Update

From Crystal:

We have a position opening in the Community Building and Network Expansion Committee in December. We really hope folks from non-US/CA/EU areas consider going for the position. Having someone from these geographic regions in a leadership role, especially for this committee, is crucial for PSA’s growth and development. I will miss being CBNEC assistant director but I am SO excited about seeing where new leadership can take this committee and the PSA in general.

If you think you might be even kinda sorta interested in this position, please DM me on Slack or email me at cnsteltenp@usi.edu! I’d love to chat about the group and brainstorm on where it can go from here!  

Gathering feedback on a possible PSA translation service

PSA member, Adeyemi Adetula, would like to solicit your feedback on potential demand for a paid translation service for African languages in terms of interest and to assess available funds to dedicate to such a service.

In general, there is little research on African populations in psychology. Even in the Psychological Science Accelerator (PSA), African participation in multi-site studies has been poor, with a recent project including just two (out of several thousand) African indigenous languages. To facilitate inclusion and improve generalizability of psychology, we propose a paid translation service for African languages and thereafter offer a more general translation service (for full details regarding this service, see here).

Who could use this service? Any researcher interested in conducting research amongst African populations.   

What do I stand to benefit? First, users would get a high-quality translation. Quality of these translations is of high priority and we intend to adopt the already established translation procedures of the PSA. This procedure required a forward translation (source to targeted language), a back translation (serving as validation), editing (reconciling text difference of first version and validation version), external reading (feedback from potential respondents), cultural adjustment, proofreading, and implementation (transfer of translated note into a survey software). Second, such a translation service allows one to conduct research in African populations as we will also try to connect users with local researchers. 

Survey. We invite you to give us feedback on this proposed service via this short (5 minute) survey (if you are instead interested in being a translator, you can fill in the survey here).

News from the Accelerator – August & September 2020

Today’s newsletter is double-sized, summarizing PSA activities from both August and September. It was a fun and busy 62 days with an election, some exciting news on publications, our first conference, and much more detailed below. As always, thank you for everything you do to accelerate psychological science. Happy reading!!

Associate Director Election Results

We had two excellent candidates in our election for an open Associate Director position: Peter Mallik and John Protzko. Please join us in congratulating John Protzko as our new Associate Director following an extremely close network vote. We look forward to seeing the vision and energy that John will bring to this new role. We also thank Peter for running, and are thrilled to continue working closely with him in other PSA roles! 

Get to know John Protozko: He is a metascientist at the University of California, Santa Barbra. He participated in PSA001 on data collection and is the PM on PSA006. His research interest is in causality as it applies to people and the implications for theory in fields such as cognitive development, social perception, and metascience. He. Loves. Food. and will travel the world for it.

PSA 2020 Conference

The first ever PSA conference is in the books, and we had tons of fun! Thank you so much to all of the organizers, presenters, moderators, hackers, and attendees for making this first meeting a success. Over the span of three days, multiple people presented studies and research on a wide variety of content, and others tuned in to listen to these talks from all over the world. If you weren’t able to make it this year, or had to miss a session, each of the talks were recorded, and are available to watch here. All videos that were successfully recorded have been uploaded (although some have been lost 😢 ).

Whether you attended the conference or not we would like to provide feedback by taking this short survey! We believe that each person in our network has a valuable opinion on the conference that we would like to learn more about as we consider the possibility of hosting another one in the future.

Study Updates

  • PSA 001: The stage 2 (final) paper was accepted at Nature Human Behaviour with only minor formatting edits required before publication!!!!!!!!!! This marks the true completion of our first ever study at the PSA.
  • PSA 002/003- The 002 and 003 teams are in the final stages of moving our experiments from Open Sesame/Qualtrics to Lab.JS. Thanks to the hard work of Merle Schuckart, we have drafts of the new programs and are in the process of ensuring that the web versions of the studies retain fidelity to the original protocols and troubleshooting the studies. The new version of the experiment should fully integrate both the 002 and 003 procedures into one online protocol, greatly simplifying future data collection.
  • PSA 004: Data collection is still going on and will wrap up at the end of the year! We do however need more CREP reviewers as we wrap up this project at the end of the year. CREP reviewers look over each lab’s OSF page to ensure that each lab is performing the study as planned. If you or someone else is interested please have them fill out this form.
  • PSA 005: We have finished our latest revision of the stage 1 manuscript based on our communications with the editor. This should be the last revision before stage 1 acceptance. The newest revisions were submitted on Wednesday!!!!
  • PSA 006: 006 will continue data collection through the end of 2020. We have collected lots of great data from all of our members and are excited to see this study through to the end.
  • PSA 007: PSA’s new project: Semantic Priming Across Many Languages (SPAML) is just getting started. Learn more by checking out the presentation from the conference: click here! The words2manylanguages part of the project is currently recruiting all kinds of help. We are trying to create a repository of datasets to use as our criterion for the modeling section of this project. We downloaded a bunch of them, but we need to clean them up to a consistent format (naming, csv column names synced, etc.). Anyone who’s interested in helping will be given credit via CREDIT (so this task would count + reviewing the paper for authorship). You can email buchananlab@gmail.com. Join slack or email to be kept updated on the two other pre-projects, more announcements on those soon. All three will need to be completed as part of the design for the full 007 project.
  • PSACR 001/002/003: Data collection is complete for PSACR 001 and 003, and PSACR 002 data collection is ending on October 23rd. A summary of data collected is available here http://formr.psysciacc.org/shiny/shiny_app/!

Director’s Statement Installments 

Chris is releasing a Director’s Statement in individual posts/installments to solicit feedback and ultimately guide priorities and activities for the remainder of his Directorship with the PSA. You can read the opening letter here and see the planned section titles with tentative release dates here!

Community Building Update

At the end of December 2020, Crystal Steltenpohl will be stepping down as an Assistant Director of the Community Building and Network Expansion Committee. Her announcement reads:

“I look forward to seeing what new energy the next assistant director will bring. I have really enjoyed working with Natalia Dutra on this committee, whose goals of diversity, inclusion, and community building in the PSA are close to my heart. Of course, even as a non-assistant director, I will still be involved and support the CBNEC. The director group is so awesome and open to feedback. I don’t usually look forward to committee meetings but the all hands meetings are fun and productive and informative. The general PSA meetings remind me of why I got into science. We have an awesome, collaborative community full of talented people from across the globe.

While I have no control over who takes my place, I really hope folks from non-US/CA/EU areas consider going for the position. Having someone from these geographic regions in a leadership role, especially for this committee, is crucial for PSA’s growth and development. I will miss being CBNEC assistant director but I am SO excited about seeing where new leadership can take this committee and the PSA more generally.

If you think you might be even kinda sorta interested in this position, please DM me on Slack or email me at cnsteltenp@usi.edu! I’d love to chat about the group and brainstorm on where it can go from here!”  

Data and Methods Update

We have some pretty exciting things happening in the Data and Methods Committee! We have added a new member to our ranks, Will McAuffille, and we are ecstatic to get to work more with him! We’ve started up a subcommittee on meta science that our very own Nick Fox is leading. Lastly, we are currently in the midst of a flurry of activity and progress on the measurement study related to PSA 006. 

Secondary Analysis Challenge

A year after the secondary analysis challenge was announced we have posted the results and submissions! We are proud to announce that it seems to have been a success as we had 8 submissions, and we learned new information from each of them. Each submission script was checked for computational reproducibility by two members of our team (Abigail Noyce and Patrick Forscher). To read more, please visit the blogpost!

IT’S THE MAP!!!

Nicholas Coles has updated our membership map and it looks sick! We continue to grow as a network day by day but it had been some time since we had updated the map. The previous map had us at 760 researchers and now we are up to 1021 and counting. Head over to the website to take a gander out how much we have grown! 

Cool articles: