4

Nick raised the question of whether some of our improving site statistics suggested we were ready for launch. I figured this might be an interesting meta discussion.

Nick: We're now in the "okay" range with 5.5 questions per day! Time to start pushing for a launch maybe? It has already been 750 days as of today.

Jeromy: Hi Nick, it's great that we've got questions per day in the okay range. My sense from these site stats stackexchange.com/sites#traffic across the network is that we're moving in the right direction, but that we're still a while off launch. Just looking, other sites that are launched typically have 10,000+ visits/day.

Nick: Wow! That is quite a while off. However, if that's your estimate for currently launched sites, might having been launched already be a causal factor in the extra traffic? That is, I assume SE isn't looking for beta sites to reach the same level of traffic as fully-launched sites before actually launching them. Area51's criteria for "excellent" status in this particular dimension certainly seem to suggest otherwise...but I also suspect that there's more involved in the decision "behind the scenes" than these five statistics, and I'm a little pessimistic of getting any insight by asking.

Question

  • How will we know that we are ready to go out of beta?
  • What site statistics might suggest that we are ready?
2
  • Thanks for promoting and answering my thoughts! With your answer alone, this is already a very useful question! FWIW, the reason I'm pessimistic about getting insight by asking on Area51 (not here from all you fine fellows!): I earned the tumbleweed badge for this question of mine on Area51, which I admit was a little intimidating in length and complexity...(most of which could be ignored if one were simply to assume the bolded questions up top deserve attention in any case!)
    – Nick Stauner
    Feb 6, 2014 at 23:08
  • 1
    This discussion might be helpful.
    – Bleeding Fingers
    Feb 7, 2014 at 20:32

3 Answers 3

Reset to default
9

Personally I think that to graduate from Beta we need to solve the problem of attracting more expert members and increasing the answer rate.

As we are, any fluctuation in membership seriously affects the site. As far as I can tell, all the very active members are comparably young, and a career step or life event such the birth of a child might take them away from the site and painfully reduce the core personell. We need a number of academically trained and active members that is large enough to keep the site running at full speed even during summer holidays.

Also, while many answers don't need more than one answer, if that answer is sufficiently "enxyclopedic", there are still too many questions that receive no or only mediocre answers. Since researching these answers can be extremely time consuming, we need more members who already know these answers, or know which book to pull from which shelf.

Just as knowing how to program C++ is not enough to answer questions about JavaScript on StackOverflow, being an expert in one field of the cognitive sciences is not enough to (efficiently) answer a question in another field. To encompass all fields, we need experts in all fields. They don't necessarily have the expertise level of publishing researchers, I think advanced students that heard a lecture on the topic are enough.

And just as Linus Torvalds does not answer questions about Linux on StackExchange, we cannot expect this site to be run by high level experts only. I think we would profit a lot, if we marketed the site more to students of Psychology, Cognitive Sciences, the Neurosciences and so on. As it is, we aim for researchers, but mostly answer questions by complete laypeople. A solid number of student members would create a middle ground from which good questions and good answers and some more intense "debate" (in the sense of posting alternative answers) could grow.

I think we're not there yet.

3
  • 1
    I agree. It's strange that we don't get more students asking questions on this site. There are so many undergraduates studying psychology out there. And it would be great if we had more experts and advanced students involved. My sense is that we get a lot of traffic from the broader stack exchange network and that that tends to be programmers and tech-type people.
    – Jeromy Anglim
    Feb 7, 2014 at 7:41
  • 4
    So maybe we'd need to think about a cost-free, not too time consuming marketing strategy aimed at students.
    – user3116
    Feb 7, 2014 at 8:09
  • 1
    @what: meta.cogsci.stackexchange.com/q/139/21
    – Steven Jeuris Mod
    Feb 7, 2014 at 9:42
2

Official words on graduation: This post on the stack exchange blog on when sites will graduate from beta is one source of information. It states:

So from this point forward, the graduation date of a site will depend heavily on having enough users with sufficient reputation to properly lead and govern the site.

Also, the point is made that:

There’s no harm in staying in public beta far beyond the initial 90 days

Inferring from other sites:

The official page of StackExchange site statistics gives a good sense of how we stack up relative to other sites including both other beta sites and graduated sites. Here's a screen shot of a rough cross-over point.

site statistics on stackexchange

Of course, there are several beta sites above these, and a few graduated sites below this. Also, visits per day is not everything. Of the graduated sites that are below, they often graduated early and feature high quality content and often many questions. If I were to guess, I think we should be aiming for:

  • 10,000+ questions
  • At least 80% answered, but higher would be better
  • 10,000+ visits per day
  • 15 questions per day
  • And a general good standard of content, moderation, and active users

But to reiterate, I don't think it matters too much whether we're in beta or not.

Does graduating increase traffic? I'm not really sure. But I don't think it would make much difference. There are plenty of beta sites with huge traffic, and non-beta sites with low traffic. The difference in site design is just a bit of polish and the removal of the word "beta". The actual content doesn't really change.

6
  • SE certainly seems to be in no hurry to graduate Personal Finance & Money, given statistics as enviable as those...One counterargument about graduation's potential effect on traffic though: don't underestimate the effects of polish and the word "beta". My tied-for-second-most-upvoted answer so far speaks to the power of polish a little bit, albeit in a rather incomplete and indirect way (pardon the shameless plug BTW). Beta status is also an implicit threat of sorts that the site may not continue to grow past some point in the future...
    – Nick Stauner
    Feb 6, 2014 at 23:41
  • ...if, when SE does make up its mind, it doesn't decide in our favor. This is a substantial concern for me as a psychologist, because I'd probably rather spend my time writing journal articles and conducting research if my work here isn't going to continue to grow in relevance without continuing effort on my part, or if my ability to provide that continuing effort here may come to an end. I don't know if others feel the same, and I'd hate to disincentivize others by spreading my own concerns unnecessarily, but I'm sure that the experts this site needs would appreciate more long-term assurance.
    – Nick Stauner
    Feb 6, 2014 at 23:45
  • 1
    Regarding "personal finance" I've heard several questions raised about the degree to which the answers reflect deep expertise; contrast that with a graduated site like "theoretical computer science" where I've heard you have some serious academics posting questions and answers.
    – Jeromy Anglim
    Feb 6, 2014 at 23:46
  • Ah yes, the inscrutable, amorphous criterion of answer quality...Small wonder we don't have site statistics automatically calculated, displayed, and judged for that! I'm afraid a behind-the-scenes criterion like that could be a particular threat here, because even professional cognitive scientists vary widely in their ability to critique others work and the thresholds they set for "good enough".
    – Nick Stauner
    Feb 6, 2014 at 23:48
  • 1
    The content is creative commons CC-BY licenced and backed up in public places. So the content wont be lost. I see no signs that the site will be closed. We have positive trajectory; it just takes time. If we got out of beta in two years, I'd be happy. However, I wouldn't stress too much about the beta thing. But at the same time I wouldn't spend so much time here that you're not pumping out papers. Which reminds me, I better get back to it :-)
    – Jeromy Anglim
    Feb 6, 2014 at 23:49
  • Ha! Good for you; my former advisor certainly agrees. I'm not worried (much less stressed :) about losing the content; just that if it ever gets archived, its content won't continue to aggregate and attract cumulative attention and structure. That active, organically evolving quality is what makes SE worth diverting my attention from the usual professional forums (i.e., journals/books/etc.), so for me personally, I'm afraid the likelihood of closure makes much of the difference in the site's value. I agree we're moving up for now though, so for me, it's a matter of planning, not pessimism. :)
    – Nick Stauner
    Feb 6, 2014 at 23:58
1

Here are some insights I've picked up today:

  • The fabled "critical mass" seems to be an inflection point in rate of change in site traffic
  • # of users with >2K and >3K rep matters for handling edits and close votes after launch.
  • Quality matters (but good luck measuring it), so don't just throw those votes around randomly either.
    • @MadScientist says quality is very important, and likely to encourage launch / prevent closure.
    • Quality probably affects user retention, which probably affects traffic accumulation over time.
    • Unanswered questions that aren't worth answering might deserve site-wide cleanup efforts.
  • The statistics Area 51 displays are a bit like a pulse reading...an index of the site's health right now
    • At least in the case of questions per day: that's a 2-week moving average.
    • Not sure about others; I've seen visits per day move around a little, but not the others.
    • % answered can matter, at least as a "saving grace"...more reason to clean up lame questions.

References

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .