Improving the System for Dealing with Poor Answers

Question

On the advice of Shog, this has been reposted on Meta, slightly simplified

So it's no shock that we have a lot of poor answers here. We are having issues with understanding the current flagging, and problems with old questions with lots of bad answers.

Kismet brought jmort, Shog9, and myself in to chat at the same time, where we had a discussion on the issue of poor answers (note: pretty long, but enlightening discussion).

Fundamental Problem

When someone asks a bad question we can put it [On Hold] to allow the poster to improve the post. If it doesn't get improved, then it will be deleted in a regular cleanup and our site is a better place for it.

There is no such option for answers.

Since graduation, we have lost a lot of people able to delete answers. That means the mods have to get involved. We are limited to voting (which doesn't have much impact when a question gets popular with lots of sympathy upvotes), or flagging (which many people seem to think we aren't using properly). This is less than ideal.

The community needs to have a way to deal with poor answers in the same way that it deals with poor questions, preferably without moderator intervention, and still giving people a chance to improve their answer prior to deletion.

Solution

We already have a great tool similar to putting a question on hold in post notices like this:

This post does not cite any references or sources. Please help improve this post by adding citations to reliable sources. Unsourced material may be challenged and removed.

Right now, these post notices can only be added by moderators, meaning that while any 3k user can vote to close a question, we are incredibly limited in our ability to do the same to answers.

Let's change that.

Proposed Workflow

1. If a question receives answers over a certain threshold, those answers are tossed in a queue for review
2. For users over a certain reputation threshold, one of the options for that queue can be to add a post notice
3. If a certain amount of users vote to add a post notice, it gets automatically added to the post without moderator action required
4. Once a post is given a post notice, if it isn't edited in a certain number of days (like the 5 for a question to go from [On Hold] to [Closed]), it is added to the Low Quality Review Queue and can be deleted regardless of score

Proposed Post Notices

I think the following post notices would be good for our site:

Insufficient Explanation

When someone just gives an answer without explaining why or how it is the appropriate answer, we can use this existing post notice:

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer: please explain why you're recommending it as a solution. Answers that don't explain anything will be deleted. See Good Subjective, Bad Subjective for more information.

Does not Cite Sources

When someone posts something about statutory or regulatory requirements (for instance, they say something is not permitted under the Americans with Disability Act) but doesn't actually cite the relevant portion of the text to allow people to confirm the answer, we can use this existing post notice:

This post does not cite any references or sources. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.

Me Too Answers

Often we have answers that don't actually add any value. They contain the same advice, same reasoning, same explanation, and just end up taking up space rather than providing value. We don't have a post notice for this, but I think we should request one be added with a notice like this:

Information in this post has already been provided in other answers. Please help improve this answer by adding additional information not already given in other answers. Answers duplicating content may be removed.

Answer 1

Two Stage Workflow

I like the workflow because it breaks things out into stages. It takes more than one person to make a call to add the post notice, then it takes more than one person to delete, with deletion only being allowed on positively scored posts if the answer had previously been reviewed and left unimproved.

The two stage approach gives answerers some time to get feedback through comments before the answers are ultimately removed. It also gives other community members a chance to edit in citations. I've put citations in other answers before to avoid deleting something that I personally thought had value.

This approach, if executed and implemented with care, could lessen the chance of abuse, while also keeping things from slipping through the cracks. There are currently 31 answers with post notices, and some are fairly old, having flown just below the radar. This plan would help ensure they're dealt with in a timely manner while still giving the community time to collaborate and resolve issues.

Are there different types of posts with different answering requirements?

The other important point that you've brought up is that there are different types of posts. On some, it makes sense to require citations and official references and for those posts to be removed if citations and references can't be produced. But there are other types of posts where only real experiences, backed up with explanations of why and how, can validate them, since there is no "official" solution to that problem.

The key is to be able to differentiate between the types of posts that need the citations and the types of posts that need the full how and why explanations. Perhaps we can break down certain tags into these two categories. Categorizing types of answers makes it more clear what guidance to give the folks answering as well as what guidance to give folks making decisions about the posts.

---

Proposed Moderator Workflow to Test the Proposed Review Changes:

Here's a proposed plan for the moderator workflow:

• On newly posted answers, community members flag as "needs citation" or "needs explanation" post notice. If you have suggestions for improving the post, please leave a helpful comment.

• Moderators evaluate and add the post notice and leave a comment, if necessary. (We pretty much do this anyway)

• If the flag is helpful, we mark it as helpful and clear it off our plate.

• Once per week, we drop the post notice answers search, sorted from oldest to newest into the Water Cooler or meta where community members review, attempt to improve, and "re-flag" the post as either "improved - remove post notice" or "very low quality" or "not an answer"_{see updates to flagging} meaning it hasn't met the guidelines for a good Workplace answer and needs to be evaluated for removal. This last part is where we typically lose sight of things and where this proposal may prove to be helpful.

At this time, what I've written is just a proposed idea. Let's hear some more feedback on this before making these adjustments.

---

• UPDATE: For the most recent flagging guidance, please see this meta post.