We are excited to announce that we have selected our first study to be conducted with the Psychological Science Accelerator distributed laboratory network.
Ben Jones and Lisa DeBruine of the University of Glasgow (http://facelab.org/) submitted an excellent proposal to test if Oosterhof and Todorov’s (2008) valence-dominance model of social perception generalizes across world regions. In their submission they explain,
“Oosterhof and Todorov (2008 PNAS) found that Principal Component Analysis of trait ratings of face images made by students at a US university produced two components. The first component, which they labeled ‘valence’, was highly correlated with rated trustworthiness. The second, which they labeled ‘dominance’, was highly correlated with rated dominance. Although this two-component model of social judgments of faces has become very influential, the extent to which it applies to trait ratings of faces made in other regions of the world is not yet known. The proposed project would use confirmatory factor analysis to establish whether the model described in Oosterhof and Todorov (2008 PNAS) can (1) be replicated in a new sample of North American raters and (2) can also explains trait-ratings made in other world regions (United Nations Country Grouping: Africa, Asia, Central America, Eastern Europe, European Union, Middle East, North America, Oceania, South America, The Caribbean).”
Their blinded submission was reviewed by over 40 members of the Psychological Science Accelerator. Our Study Selection Committee found it feasible for our initial efforts, our Advisory Committees noted many strengths of the submission and the likely impact of such a study, and we ultimately decided it was an excellent study to kick-off the Accelerator!
In the coming days and weeks, all experimental materials, protocols, translated instructions, and analysis scripts will be finalized in a collaborative effort between the proposing authors and our committee members. We look forward to subsequently matching laboratories from our network with this exciting project.
While we will invest considerable data collection in this study, it will not come close to exhausting the overall data collection capacity of the Psychological Science Accelerator for 2018. Thus, we continue to review the other excellent and exciting submissions that we received following our first call for studies. More announcements will be coming soon!
If you would like to join the Psychological Science Accelerator, to assist in data collection for this specific study, or to be involved going forward, please sign up here to receive more information!
“The proposed project would use confirmatory factor analysis to establish whether the model described in Oosterhof and Todorov (2008 PNAS) can (1) be replicated in a new sample of North American raters and (2) can also explains trait-ratings made in other world regions (United Nations Country Grouping: Africa, Asia, Central America, Eastern Europe, European Union, Middle East, North America, Oceania, South America, The Caribbean).””
This doesn’t make much sense to me. “United Nations Country Grouping” ?! Isn’t this a geopolitical division? Is that the most appropriate division that can be made, keeping in mind the possible working mechanism, possible origin of the effect, and relevant literature on this topic? I highly doubt that.
It wouldn’t surprise me if people in different countries within your proposed division have different trait-ratings. It wouldn’t even surprise me if different groups of people within different countries have different trait-ratings. This could have to do with several things: socioeconomic status, ethnicity, gender, etc.
I don’t know what the several things are that you are going to measure/record, but i would be interested in viewing what variables, other than country, you are planning to measure/record. I couldn’t find a link to the proposal in this post, can you provide a link?
It seems to me that, from a scientific perspective, it would seem useful to try and explain the effect and possible differences. To do this, it would seem useful to me to 1) measure/record the several things that could account for the effect and possible differences, and 2) only then see how these variables compare and possibly make groups according to shared variables.
It seems to me that you at least are planning on measuring/recording countries. It would seem more appropriate to me to first measure/record the country, and then possibly check for groups that can be formed using different countries based on similar ratings. You seem to be wanting to do this the other way around, which i find rather strange reasoning from a scientific perspective…
Thank you for the thorough and thoughtful comment! A quick response for now. First, many details of the study remain open for debate and await decision in collaboration between the proposing authors and members of the Accelerator. Second, we can’t yet share the proposal…they were designed for a very specific purpose (review but they Accelerator) and not for public dissemination. BUT, we will be sharing more details of the project as the emerge. Secondly, we hope that the data sets produced by the Accelerator (which will be open by default) will be a rich resource for secondary analyses such as the ones you describe. Third, for a study to be truly confirmatory in nature, as is the goal for this specific study, waiting to decide grouping variable of interest until one is analyzing the data is problematic for many reasons. This proposal has a relatively narrow and confirmatory scope, but should provide an excellent data set for exploration and further hypothesis generation!
“Thank you for the thorough and thoughtful comment!”
I always worry this is academic language for “your comment does not make much sense, but thanks for trying” 🙂
Regardless, it allows me to make my point in a slightly different way which i hope is okay, and more importantly, might be worth considering.
I took a look at your starting page where you talk about inclusion, decentralized authority, transparency, rigor, and humility. I like these things in science, and i like them even more regarding large, possibly impactful, collaborations. These values/principles (which i think can be perceived as being scientific values/principles) could perhaps also be kept in mind viewing Psychological Science Accelerator from an even wider perspective.
If i am not mistaken, you may already have had some possibly useful feedback via twitter (and perhaps other sources), which possibly underscores the point i am trying to make: the idea behind inclusion, decentralized authority, transparency, rigor, and humility can perhaps not only be scientifically useful for the labs that are participating, or the 40 members that chose the study, but can also possibly be extended even further.
You could think of having psychologists vote for a top 10 of studies (cf. Grahe et al., 2012) they would like to see performed. And, you could make the design of the research protocol (that will subsequently be executed by the participating labs) even more inclusive, and transparent, by putting it online somewhere where people can comment on it, so it can possibly be improved. Etc.
That’s about as much as i think i can try and contribute. I thank you very much for giving me the opportunity to try and contribute something useful via your choice of having a comment-section on this blog. |And i thank you for your reply!
Haha, no! Your comment made perfect sense to me. We’re already planning ways to get more “outside” input on our general practices, and also on specific studies. I like your thought of having a much larger group of psychologists vote for important data collection projects. For this first project, we actually do plan to post all materials as we are developing them. We’re on a pretty tight timeline on our first (we promised proposing authors we would collect data starting early in 2018) so I’m not sure how much the protocol can change based on external feedback. This should be a bigger priority of our going forward, in my opinion. In essence, we want this initiative to belong to the whole field, and not just those in the network. With that in mind, I’ll be hosting a small google hangout this Friday 14:00 UTC to get some feedback. I’m reserving 4 seats for network members and 4 seats for others. Feel free to shoot me an email if you’d like to join (cchartie@ashland.edu)! Cheers!
Hi there,
What a great first study – I’m very interested in seeing how universal the dimensions appear across cultures.
Is there a proper way that I am able to cite the proposal for your study, or to cite Psych Accelerator?
We are hard at work on a paper to more formally introduce the project, but for now, the blog posts are the best citation. Thanks!
“We are hard at work on a paper to more formally introduce the project (…)”
I wondered if the following might be interesting, and useful, for you guys to investigate concerning 1) how to optimally accelerate psychological science, and 2) your possible paper about the project:
1) Take a look at all the Registered Replication Reports (RRR’s) performed thusfar
2) Randomly take 1, 2, 3, 4, 5, etc. individual labs from one of these RRR’s and their individual associated confidence intervals, effect sizes, p-values, no. of participants, etc.
3) Compare the pooling of the information of these 1, 2, 3, 4, 5, etc. individual labs to the overall results of that specific RRR
4) Try and find out what the “optimal” no. of labs/participants could be to not waste possible unnecessary resources
5) Possibly use this information to support the no. of labs/participants per PSA-study in your paper, and/or use the information coming from this investigation to come up with a possibly improved manner to optimally accelerate psychological science.
How is the Psychological Science Accelerator’s First Study?
Going well! We’ve collected data from over 10k participants, and plan to wrap up collection in September. Dissemination soon after!
I’m very interested in seeing how universal the dimensions appear across cultures.
Hi there,
What a great first study – I’m very interested in seeing how universal the dimensions appear across cultures.
Is there a proper way that I am able to cite the proposal for your study, or to cite Psych Accelerator?