What to do when someone calls your research BS

By now I’m sure you’re all aware of the controversy surrounding Amy Cuddy’s power posing research. For a great history of the debate, see here:  https://www.nytimes.com/2017/10/18/magazine/when-the-revolution-came-for-amy-cuddy.html

Now, when a scientist hears the phrase ‘power-posing,‘ they all cringe, and some have outright labelled the research as “crap.”  I recently ran a study using Galinksy’s (Galinsky, Gruenfeld & Magee, 2003) manipulation of power (recalling an event in which you have power over someone, or someone has power over you) in the context of gender discrimination, which showed that power played a role in responding to discrimination. Before writing it up for publication I wanted to replicate the finding and considered adding in the power pose, exactly as Galinsky and colleague’s had done. They cross the recall of power with the power pose, showing that both manipulations had different effects on different outcomes (Huang, Galinsky, Gruenfeld & Guillory, 2011). My students were quite excited about this possibility, and then they heard about the treatment of Amy Cuddy, and weren’t sure they wanted to be exposed to the ridicule that she had experienced.  So, I decided to make this a teaching moment about how to, as a colleague of mine put it, “stand up for yourself in a scientific debate.”

 

  1. If someone ever says to you (as many have said to Amy Cuddy), “oh, that topic is BS” with no supporting evidence, just an outright declaration that ‘the King is Dead’, perhaps suggest to them that such an attitude is an example of what not to do when talking about science…Although the media incorrectly often makes conclusions based on one study, we as scientists should let the data speak for itself by basing conclusions on literature reviews and/or meta-analyses (summary papers that assess the effects of a topic over a multitude of studies). What meta-analyses often reveal is that most phenomenon we study in psychology will be replicated under some conditions but not others, or have an impact on some outcomes but not others. And in fact, the presumably ‘unworthy topic’ of power posing is no different—while some studies could not replicate the effects in certain conditions on certain variables, other studies have successfully done so (e.g., Carney et al., 2015; see also https://docs.google.com/spreadsheets/d/1VZQxTNGncn-x7nz9OsNXmkz9rFkhdYEjzNXN7vqrYKA/pubhtml?gid=1181532305&single=true)

 

  1. You could try to spark a discussion about the debates on what is publishable data. Some call for only replicated studies to be published as there is greater confidence in those findings; some call for any well-designed study with a sound theoretical rationale to be published, because if only significant and replicated studies are published, we may obscure accidental findings (e.g., Viagra was supposed to be a treatment for angina), and devalue exploratory research. Indeed, if the discipline had valued non-significant findings as well as significant findings, perhaps all the data would better speak for itself.

 

  1. You could highlight gender differences in the treatment of scientists. Contrast how Amy Cuddy has been treated with how top male social psychologists like Daryl Bem (who was able to publish work on Extra Sensory Perception that also had extremely low sample sizes in a top-tier journal) or Adam Galinsky (who along with colleagues, has published on power posing itself; see Huang et al., 2011) have not been ostracized. Further to that, Galinsky et al.’s (2006) finding on the effects of his power manipulation using recall was not replicated by Many Labs 3 (Galinsky, Rucker & Magee, 2016), but he did not suffer the same ostracism that Amy Cuddy endured when her 2010 finding was not replicated.

 

  1. After making those arguments, someone may rebut that, “well, my attitude is not different than even a co-author of the paper herself.” Indeed, a co-author of the original paper has equally decreed that power posing should not be studied (http://faculty.haas.berkeley.edu/dana_carney/pdf_My%20position%20on%20power%20poses.pdf ). In response, you could integrate other literatures to argue that this decree is not necessarily based on science, but perhaps it is an understandable need to distance oneself in the face of threat (e.g., Kemeny, Gruenewald, & Dickerson, 2004; Murray, Holmes & Collins, 2006).

 

  1. And finally, you could use this as an opportunity to discuss academic freedom and whether under any circumstances, is it ok to “discourage others from studying power poses” (see link above); as scientists, it might do us some good to wander back into history (e.g., Galileo’s condemnation by the Church for saying the earth was round; Nazi Germany’s mandate that only approved party topics be studied in school) to consider the implications of telling others what deserves scientific inquiry.

 

As the advisor of a students who were originally excited about studying a particular topic, but are now fearful of how they’ll be perceived, I’m forced to decide between standing up for the principles of real science (following the scientific method, then letting the data speak for itself) and doing the research or not doing the research, thereby protecting my students from the unscientific ostracism.  In today’s climate, I will probably choose my students over what I think is right. But I’m not sure how right that is.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s