10

So I've just come from a conference attended by psychologists who share an interest in concerns about the replicability of the field's findings, as well as considering what may be done to improve it.

Repeatedly, conference attendees highlighted the need for a Q&A resource where they would be able to ask questions, get answers, and learn about aspects of the replication "crisis", open-science, and meta-science, thereby generating a lot of methods-focused discussion.

Naturally, I thought, "this sounds perfect for a Stack site". After finding the Cognitive Sciences Stack though, I'm not quite sure what intended "target market" of users is these days, as from what I've seen in other posts on the meta Stack, it seems as though this may be a period of transition for the site.

I wanted to therefore check in and ask:

Are questions on these sorts of topics (replicability of psychological research, open-science methods, meta-science) considered as falling within the purview of the Cognitive Science site?

For example, someone might ask for an explanation of current methods used to quantify the replicability of psychological literatures, and how they differ from one another. A good answer might include accessible descriptions and comparisons of methods like p-curve (Simonsohn, Nelson, & Simmons, 2014), R-Index and TIVA (Schimmack, 2014), statcheck(Nuijten et al., 2015), and GRIM (Brown & Heathers, 2016), and might even point to a user-friendly resource like p-checker (http://shinyapps.org/apps/p-checker/) that helps psychologists use these methods themselves.

Some other hypothetical questions of this sort could include:

  1. Asking about the merits/concerns of pre-registration of hypotheses, materials, and data-analysis plans, as well as recommendations for pre-registration services.
  2. Asking for an explanation of how various meta-analytic methods attempt to assess and correct for publication bias in the psychological literature
  3. Asking for a summary of evidence supporting/challenging the impact of the adoption of open-science practices in the field.

Though some of these questions (like #2) and others might also be relevant on other Stacks sites (e.g., CrossValidated), the issues seem especially relevant to psychologists as our field seems preoccupied with concerns about replicability and building a cumulative science.

If questions like these are considered on topic, I suspect there a number of psychologists that would enjoy using this site as resource to learn about those issues (and I and others from the conference could help to draw attention to this site, for that purpose).

  • Could you give some focused examples and bold-out the core of your question? I don't really seem to grasp where you are heading here, especially so given the answer of Steven below. Your question couldn't come more timely, given the current developments of this site. – AliceD Jun 8 '16 at 19:37
  • 1
    Sure--updated my answer with more content and some of the formatting you asked for. – jsakaluk Jun 8 '16 at 22:12
  • Thanks! I'll go through your question more thoroughly later +1, but for now I'd like to point out Academia.SE for broad questions that only touch upon CogSci. – AliceD Jun 8 '16 at 22:15
  • 2
    Yeah, I am aware of Academia--I just don't think that is a topically focused enough forum for the needs of the community of scholars I am referring to. Anyways, I'll look forward to seeing what the Cognitive Sciences community thinks about the inclusion of this topic. – jsakaluk Jun 8 '16 at 22:17
  • 3
    Yes, please! As long as the questions touch on how these areas impact the psychology literature, they should certainly be on topic here. And I would argue that any discussion of methods that are related to the psychology literature should be on topic as well. – Josh de Leeuw Jun 9 '16 at 14:12
8

The transition we are currently going through is whether or not the site should be made exclusively for experts. There are no people who argue experts are not welcome here, thus by all means, what you describe is very much on topic here!

Glad you found this site, welcome!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .