I think the object is to provide correct answers. The fact that an answer has passed the "peer review" of intelligent people like ourselves should give credence to the fact that it is indeed correct to the best of our understandings.
If there is a particular point in which someone has doubt or it is suspected that someone else may have doubt, that point should be brought up in comments.
For example: "You're making the claim that 80% of widgets are made in China, how do you know that to be true? I mean, I could personally agree with you, but there may be some people who aren't sure about that."
Or maybe, "I noticed your claim that 80% of widgets are made in China is unsubstantiated, so here is a link I thought you might find useful in your answer to back-up your claim."
But instead what I often see is "Boo! Bad answer! No references! You stink! -1! I hope when you get home your momma runs out from under the porch and bites you on the leg!" Ok, that's a little dramatic, but you see where I'm going with that.
This site is so competitive that folks aren't eager to help each other, but rather they attack each other.
References are more often wrong than common sense.
I suppose you want me to reference that claim, huh.
Here is a university claiming you should water your lawn "deep and infrequently"
It is desirable to keep the interval between waterings as long as possible without allowing the plants to go into water stress. Deep, infrequent irrigations cause plants to develop deep, strong root systems that can extract water from a much larger volume of soil than the shallow roots associated with light, frequent irrigations.
Here is a university claiming you should water your lawn "light and frequently"
Light, frequent applications of water are much more productive than heavy (soaking) applications once a week. Remember that turf roots are naturally shorter during hot and dry weather and water moved past the root zone is wasteful and of no benefit. Research at Michigan State University indicates that damage from certain turf diseases and insects are reduced when light, frequent (daily) irrigation is used compared to a heavy, infrequent irrigation.
So who is right? Only common sense and logical reasoning can reveal it.
You can often find a "reference" to back whatever claim you make.
You want to traditionally claim women are better multitaskers? Ok, I have a reference for that:
We found that men's performance in a commonly adopted cognitive test deteriorated when they were required to coordinate this primary test with a simple secondary test. In sharp contrast, women's performance on the very same test improved upon the inclusion of the secondary test.
Oh, you want to controversially claim men are better? Well, I can help with that too:
In both experiments, males outperformed females in monitoring accuracy.
It seems almost anything can be spun using references. I often see answers voted all the way to the top and are accepted, yet are completely wrong. But they have references, so its all good!
By the way, what does the asker of a question know about "accepting" the right answer? If I ask a question and one of you answers it, how do I know its the right answer? I'm the dummy here, you're the pros. If I were in a position to judge correct answers, I wouldn't be asking questions. But that issue is tangent to this topic. I just wanted to throw that out there.
Some useful things can't be referenced
Common sense is one. If I make the claim that antibiotics are for the treatment of the underlying cause of illness rather than for the treatment of symptoms, I wouldn't expect to have to dig up a reference for that. Its common sense because its common for people to know what antibiotics are and reasonable to expect most people to know what they do.
Opinions can't be referenced. If I say "I think women evolved to have a multitasking advantage because they had to tend to multiple children while doing necessary chores. Yet men evolved to have a lot of focus which is needed in hunting and fighting." Obviously, I can't reference that except with a reference to someone else making the same conjecture, which isn't proof of anything except 2 guys thinking alike. Does it mean its not useful and worthy of consideration?
Anecdotal evidence. "I've noticed my tomato plants are healthier when I put lime on the soil." Well, I suppose I could post a study which says the same thing, for instance: "Our results indicate the application of lime decreased the incidence of blight X%". Or I could post a study which says the results are inconclusive. Now, I understand that a published study carries more weight than my anecdote, but sometimes we have no choice. Its either answer with an anecdote or no answer is possible. Sometimes an anecdote adds credence to the studies presented. See Anecdotal value
The reference issue has gotten way out of hand in my opinion. Answers should be judged on their merits of usefulness and not on the strict, anally-retentive adherence to some predetermind format. I think this is especially true with psychology because not a whole lot can be written in stone, as opposed to math or physics. Many answers will be "I think.." or "I've noticed.." and to exclude them is to stiffle progress to further discovery.
As an illustration of that point (and to echo Jeromy's), there have been many questions on cogsci that I have gotten an answer typed out, then decided not to submit it because "I didn't want to go out on that limb". Not to toot my own horn, but I feel that is a loss to the community. As I'm sitting here now debating if I should click the button... or stay out of this debate. Well, curiosity got the better of me.
I believe answers should be judged like this: 1) Does the answer answer the question or help in any way at all with answering the question? 2) Is the answer true as best you can tell? 3) Is the answer useful? If the answers to these 3 questions is yes, then upvote and be on your way. If its no, then leave a comment on how the answer can be improved. If the author refuses to improve the answer, then you have no choice but to downvote.
I don't believe answers should be judged like this: 1) Does this answer have references? If yes, upvote. If no, downvote and leave nasty comment.