3

TLDR: Many questions are not reviewed by enough users to accumulate sufficient votes to close, and as a result, what few close votes are cast, end up aging away and are effectively wasted.

I expect that it can be frustrating for users who take the time to cast close votes to find that questions don't get closed simply because no one else looks at them. It wouldn't surprise me if some users have stopped using the review queue because of this, compounding the problem. This concern has been raised before, so my intention with this post is to provide much more detail, and concrete options for moving forward.

Background

I did not know much about how the review queue works before looking into this, so here are some details:

  • Users with at least 500 rep can vote to close or reopen questions.
  • In general, 5 close votes are required to get a question closed, except for moderator votes, which are binding.
  • The first close vote adds a review task to the close vote queue, and it remains pending for up to 14 days.
  • If the review task expires, then the close votes also "age away" (invalidate), which means that any future close vote starts the process anew. The reason for this is because if close votes did not age away, then eventually all questions would get closed - given infinite time, they would eventually accumulate sufficient close votes.
  • Note that task expiry happens silently.

Statistics

We can get a rough idea of the health of the review queue by looking at the average number of users that go through each review task:

enter image description here

Note that averages are below 5 partly because some percentage of tasks are completed by moderators, while others expire. Assuming that moderator intervention is more or less constant, the differences are largely due to user activity. Averages over the past 2-3 years indicate that most review tasks do not have sufficient reviewers to complete without moderator intervention.

Options

There are several options available to low-volume sites like ours:

  1. We can ask to have our site require only 3 votes to close questions instead of 5. Since the close vote privilege is based on rep, rather than the knowledge or trust needed for this task, there is a risk that question closing may be influenced by a small group of unqualified users. Moreover, many review tasks currently don't get even 3 reviews before aging away.
  2. We can ask to change the grey circles to red. Currently, review queues get a grey circle when there are tasks pending, and turn red when there are many tasks pending. Making the queue red whenever there are any pending tasks has apparently helped some other low-volume sites get more attention to their review queues.
  3. Moderators can manually monitor for tasks about to age away using the close votes tool, or search for tasks that have already expired using the data explorer, and cast their binding vote if they agree with existing votes. I personally have started doing both of these tasks, but hammering questions with aged away votes has caused problems before, so the trick is to close just before votes expire...
  4. I have posted a proposal on MSE to warn before close votes expire, giving both users and moderators a chance to rescue these tasks, if appropriate. My preference is to let users have their say before hammering a question, but I also would be happy to intervene with my binding vote if I knew it was needed. While I think this solution would be ideal, even if it was accepted, it may not be implemented for years.

If you like any of these options, or have your own to suggest, then post an answer accordingly, and we'll see how the community votes.

10
  • 1
    The 3-vote criterion is the best way to go; I thought this would be implemented SE wide? I was still in the process of waiting for it to go through as the 3-vote criterion's made its way out of beta afaik?
    – AliceD Mod
    Jan 11 at 8:32
  • I have had more dealings with upset users over at Bio, that site is much bigger and the close vote queue there is sometimes >65. Every now and then I run (well... ran) through the whole list, and given it's a lot of work I tended to become trigger happy at times. So I stopped doing mass votes altogether.
    – AliceD Mod
    Jan 11 at 8:33
  • 2
    I believe the 3 vote queue will be a "you have to ask for it" feature after mod feedback on some sites was to not change it (Bio was one of the sites where the community was strongly against it).
    – Bryan Krause Mod
    Jan 11 at 14:34
  • One complicating factor is that it's quite possible to review outside the review queue. On this site I see every new question posted (there aren't that many!) and "review" the majority of them without ever visiting the official "queue". Not sure how common that is, and not saying there isn't a problem, just that the # reviews per task is not necessarily equal to the number of people viewing the question with close vote powers.
    – Bryan Krause Mod
    Jan 11 at 14:37
  • @BryanKrause Exactly. I personally don't use the review queue. I regularly check the front page as it represents "current activity" and simply moderate that. But, maybe I should consider looking at the queues more. :/ I just tend to open full questions regardless since I prefer to see the full context before making decisions.
    – Steven Jeuris Mod
    Jan 11 at 16:10
  • 4
    @StevenJeuris Besides the review queues I've found psychology.stackexchange.com/tools?tab=Close to be useful as a moderator, particularly on Biology, primarily to see questions with 3-4 close votes where I previously abstained from using the unilateral mod closure but might feel more comfortable being a 5th "just like a normal user" vote or perhaps a "my vote counts like two rather than five" vote.
    – Bryan Krause Mod
    Jan 11 at 16:27
  • I second the 3-vote close @AliceD pointed out. It is being discussed in MedicalSciences.Meta and seems to be a popular idea. I cannot understand why it wasn't implemented SE wide if such a huge site like StackOverflow were to get it Jan 14 at 14:45
  • 1
    As for the review queues, I have been purposely clicking on the review queues regularly and getting nothing for a long while. Maybe because I review every post, pretty much, as they come in. Jan 14 at 14:47
  • I like the red review circle idea too as it definitely would grab attention more. Those who don't know about it may think, "oh, what's that?". Jan 14 at 14:57
  • 1
    @ChrisRogers Some moderators and communities were opposed to applying the change to their specific sites, and SE has been working towards being more responsive to individual communities, especially after some high profile disconnects between community and company over the past few years.
    – Bryan Krause Mod
    Jan 19 at 16:39

You must log in to answer this question.

Browse other questions tagged .