How big is too big for questions?

Question

Due to the nature of our site, many questions can be really long and difficult to answer. At some point, we need to close questions as too broad and request that they be broken up into smaller, more specific questions. As a community, we need to decide on some specific guidelines for how large questions can be. Is if one question per Question? Or a narrow topic? Or do we deal with it on a case-by-case basis?

This is necessary both to help new users know how specific to make their questions and also so that us moderators have a policy we can follow about when to close as too broad and when to leave open.

So how big is too big for questions and how do we determine reliably when to close and when to leave open?

Answer 1

Some thoughts:

There is no reliable way and it will always be a hunch. Some questions are hard to nail down and need a bunch of subquestions to define them. Others only need their title and even filling in the text field below that is unnecessary redundancy. But even those questions that need a fence of subquestions to circumscribe them all point at a single focus of interest.

If a question reads like an essay and asks you to cover all aspects of a field in your answer, it is too much. I think Taal's questions on hypnotherapy and NLP are good examples of this. I'd love to see a poll asking who of the 111 visitors actually read through the whole question. A question that makes you want to go tl;dr, is too big. The problem is not that any part of the question is wrong or off-topic, but that Taal wants to understand everything about hypnotherapy and NLP through one single question. And that is not what SE can do, that would be a chapter in a psychotherapy handbook, and those chapters exist and wait in the next university library for Taal to read.

My personal preference in natural sciences is: one question per question.

In psychology, journal articles have a scope of one question per article. My reason for that is, that after an answer, there are many open paths that your research might take. If you try to foresee all the related questions, you are preliminarily closing all those open doors, denying you the chance to have surprising ideas and make unexpected discoveries. How do you know what your next question will be after you have heard the answer to your first? And even if you still want to know the same thing, why not give your conversational partner a chance to chime in? Fun science is a dialogue, not a series of mind-numbing monologues. A question that is boring you while you read it, or makes you impatient with wanting to answer to the first sentence, is too big.

Answer 2

IMHO, the difficulty of answering succinctly is the answerer's problem, and the reader's problem. Writers need clarity and efficiency; readers need patience, motivation, and skimming skills. See my answer to this question to get some sense of where I'm coming from on the matter of complexity, length, style, and responsibility. As for the OP's responsibilities, I think we should focus on defining the banner text as is:

There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.

Too many possible answers is hard to disambiguate from the "primarily opinion-based" banner text:

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.

It's hard for me to imagine "too many possible answers" that would be based on facts, references, or specific expertise. I feel that we want answers to questions like these, e.g., "Why might people choose to get divorced?" but we don't want to feel obligated to provide every available answer, and we shouldn't. Therefore we could close a question like this for being too broad, or we could choose to add our own answers pertaining to whatever we feel is missing from answers so far. I don't see any harm in taking the latter approach, whereas I do see harm in the former. I would welcome any counterpoints on this opinion.

A more legitimately too-broad question is one that explicitly demands a complete answer or more answers than people can be expected to provide, such as, "Please list all the articles that cite Freud's (1915) The Unconscious." This question is not off-topic per se, nor opinion-based, nor unclear, so "too broad" seems like the right problem to raise with a question like this; no one wants to do someone else's literature review if it's not sufficiently focused.

This leads to lesser and more ambiguous varieties of the same problem, where the question is clear and focused on empirical cognitive science, but not a reasonable subset. I'm inclined to interpret "reasonable subset" pretty liberally, but I know others aren't, so expect this will have to remain a largely subjective, democratic judgment process. Fortunately, five votes are required, and people can always edit. I strongly encourage commenting in any case where the judgment feels even slightly subjective on the part of the voter, or at least voting on a preexisting comment that expresses the voter's opinion if it's already been stated in a constructively critical manner.

A relatively objective anchor for these judgments and a potential pitfall of liberal tolerance of broad questions is that, even if these can be answered partially and objectively, questions that are too broad are unlikely to receive answers that the OPer would accept with the check mark and +15 rep bonus. The SE system wants OPers to accept answers eventually; it's a low priority, but non-negligible, and related to the breadth problem. If you think the question is worded in a way that makes a truly acceptable answer almost unfathomable, the question might be too broad. If you think your question will come across this way, or someone tells you that your question is too broad, it might be wise to include a little disclaimer saying you'll accept less-than-comprehensive answers. IMHO, people should feel free to provide less-than-complete answers in general, though we should probably at least acknowledge this choice explicitly. Doing so takes the pressure of a too-broad question off the answerer, and allows an incrementally useful response to be made, thus making the question useful. If the OP can be convinced to accept an answer after a reasonable amount of time, and a sufficiently decent attempt to answer it usefully has appeared, then I really don't see the harm.

IMHO, "Good answers would be too long for this format" seems inapplicable here on Cognitive Sciences. On other SE sites, sure, but the character limits here aren't bad (30,000 per answer). I managed to hit the limit myself on my first version of this answer, but it was a very long, deliberately overzealous answer, and turned out to be too long to comfortably fall within fair usage rights for legal purposes. If you take a glance, you can get an idea of just how much effort it takes to become literally too long for this format. Bear in mind that a great number of the characters in this answer were used up with code and hyperlinks, so a pure text answer could've been much longer still.

I have very little sympathy for readers who opine, "TL;DR" on matters of cognitive psychology. Scientific journal articles (let alone books) are far longer than our answers here; my dissertation exceeds the character limit by almost 400%. This is why we skim at least a little bit unless we're deeply interested in every detail and nuance. My instructors have had limited sympathy for the amount I can reasonably expect to read, so I've gotten pretty good at skimming. I skim practically every answer I "read" here, and highly recommend it. Therefore, what kind of answer would be too long for this format is a separate question in my opinion. I can't think of a clear question of empirically-based cognitive science that couldn't be answered well in far less than 30,000 characters.

Despite the implications of this paragraph, I think good answers will tend to be succinct in general, or at least provide a succinct summary, even to incredibly broad questions like, "What is consciousness?" This is the intrinsic nature and challenge of work in our field. Hence I do not believe even this question is too broad. Wikipedia's intro on the topic is less than 10% of the character limit, and it's quite a good answer to such a question, which is more problematic for lack of evidence of original research. If we feel the need to close questions for lack of original research, I do not think we should do so by voting to close as "too broad"; this would be misleading.

"Adding details to narrow the answer set or isolate an issue that can be answered in a few paragraphs" is good advice, and may be the most usefully clear content in the "too broad" banner text. I don't think we need to close questions to provide this advice, and I'm disinclined to enforce it as a rule, not just advice. Still, in the truly excessive cases described above, I can see and respect reason for disagreement, and am of course willing to participate supportively in the democratic process. In some of these cases, I too would vote to close as too broad...but in most cases, I will probably find other reasons to close more useful.

This is not to say "too broad" is useless – again, for example, it is the best way to close a question requesting an excessively broad reference list that comes to my mind. I am only saying that "too broad" is probably the least appropriate reason to close most problematic questions here. Most broad questions about cognitive psychology are manageable nonetheless. Exceptions will be rare by my standards. If this seems unreasonable or unclear, please comment. I would be happy to consider any examples we have of questions closed as too broad.

For instance, I agree that @what's example of the NLP question is too broad, but not for its length. It is somewhat unclear for its length and lack of attention-organizing formatting, so I can understand the downvotes (I might not downvote it myself...but I wouldn't upvote it), but what makes it too broad are the demands placed on answers. I find it inconsiderate to demand depth, citations, and analyses, and to rule out focusing on individual aspects. This is the problem I referred to above: explicitly demanding a complete answer or more answers than people can be expected to provide. (For free!) Ain't nobody got time for that.

[Edit]: Found another good example of a question that even I think is simply too broad as is: https://cogsci.stackexchange.com/q/5902/4086

As men prove their instability on history, what would happen if all men on the earth die? Would the remaining women continue violence/war accross the world? Are women as dangerous as men when they have power? Please take diverse effects of ovulation into account.

The problem of explicitly requesting incorporation of a broad field of related(?) info occurs again in the last sentence, but the unique problem is the speculativity of the first two questions. Of course, mixing them with the third doesn't help, but IMHO, the third question is manageable (if lacking evidence of initial research effort). In some sense, so are the first two questions – I don't think answers would necessarily, inevitably be opinion-based – but "what would happen" in an apocalyptic scenario is an unclear question, and would probably remain too broad even if that weren't the case. I don't especially mind limited speculation of the sort the second question requests, but I think the first one at least needs to be reworded more specifically and as a premise rather than a question.

Answer 3

I came across this question when reading Taal's question on Hypnotherapy and although it is covered here that asking about everything to do with a subject in one question tends to be too broad, when you look at Taal's question, it's not just looking to understand everything.

There are all sorts of reasons why this question is too broad.

1. it is asking for the answerer to find all the relevant literature to cite, read it through, quote the abstracts, detail their methodologies, conclusions and findings, then evaluate the literature on its accuracy.

   Citation/Research Wise: >

   • I'd like to know if its backed by literature. I want the specific citation's abstract quoted.
   • I'd like to know their findings, conclusions, and methodologies and your analysis of that.
   • I'd like to know how long the treatments lasted after they stopped being hypnotized.
   • I'd like to know how effective it was for whatever specific goals that it was used to treat. How long did the affects last if there were any?
   • I'd like to know if you found any potential errors or flaws in the study - and you're overall analysis of that.

2. It is looking at all possible psychological disorders when hypnotherapy has no efficacy for some disorders so therefore different results can be gained in each possible enormous answer

   It can be any disorder of any kind, just as long as someone could classify it as a psychological "problem" in some context.

3. In order to answer fully, not only do you have to carry out and write all their own research for them, you need to have undertaken some hypnotherapy sessions yourself and evaluate its effectiveness.

   Anecdotally:

   • I'd like to know what it feels like to be hypnotized if you've had the experience of it.
   • I'd like to know why if it worked really well for you then why did you stop doing it or have you actually continued?

This is way too much to expect of anyone here.

It all boils down to the motivation of the person asking. Anyone asking questions here should be putting their own efforts into learning the subject. They need to be willing to read and evaluate literature cited for themselves and where necessary, ask further questions here afterwards if they don't quite understand what they have read or have all the necessary information they need to understand the topic.

My way of thinking is this:

• If the question is looking for an answer which is so long it could fill an entire book or section/chapter, then vote to close for being too broad.

• If the question is looking for the answerer to read and evaluate every piece of literature available on a subject like Taal's question, then vote to close for being too broad as it is an unreasonable request for a site such as this.

The original Too broad vote closure reason is actually relabelled by StackExchange as Needs more focus now, which causes a bit of an issue with these situations. So, I proposed in a separate meta discussion, a separate "community specific reason" of too broad for this purpose.