4

It's New Year's Day in Stack Exchange land...

A distinguishing characteristic of these sites is how they are moderated:

We designed the Stack Exchange network engine to be mostly self-regulating, in that we amortize the overall moderation cost of the system across thousands of teeny-tiny slices of effort contributed by regular, everyday users.
-- A Theory of Moderation

While there certainly are Moderators here, a significant amount of the moderation is done by ordinary people, using the privileges they've earned by virtue of their contributions to the site. Each of you contributes a little bit of time and effort, and together you accomplish much.

As we enter a new year, let's pause and reflect, taking a moment to appreciate the work that we do here together. And what could be more festive than a big pile of numbers? So here is a breakdown of moderation actions performed on The Workplace over the past 12 months:

                  Action                  Moderators Community¹
----------------------------------------- ---------- ----------
Users suspended²                                  19         29
Users destroyed³                                   9          0
Users deleted                                      5          0
Users contacted                                   52         11
Tasks reviewed⁴: Suggested Edit queue             49      4,051
Tasks reviewed⁴: Reopen Vote queue                 0      2,623
Tasks reviewed⁴: Low Quality Posts queue          14      1,139
Tasks reviewed⁴: Late Answer queue                 0        152
Tasks reviewed⁴: First Post queue                  0      4,364
Tasks reviewed⁴: Close Votes queue                 4     12,029
Tags merged                                       10          0
Tag synonyms proposed                              1          2
Tag synonyms created                               2          3
Revisions redacted                                 2          1
Questions unprotected                              0          2
Questions reopened                                10        162
Questions protected                              109        227
Questions migrated                                 8          2
Questions merged                                   2          0
Questions flagged⁵                                11      3,194
Questions closed                                 189      1,642
Question flags handled⁵                          791      2,410
Posts unlocked                                     2         25
Posts undeleted                                   18         98
Posts locked                                      16        318
Posts deleted⁶                                   237      2,182
Posts bumped                                       0         10
Escalations to the Community Manager team          8          4
Comments undeleted                               142         12
Comments flagged                                  70      9,712
Comments deleted⁷                             12,938     13,821
Comment flags handled                          5,769      4,016
Answers flagged                                   32      2,032
Answer flags handled                             960      1,101
All comments on a post moved to chat             292        115

Footnotes

¹ "Community" here refers both to the membership of The Workplace without diamonds next to their names, and to the automated systems otherwise known as user #-1.

² The system will suspend users under three circumstances: when a user is recreated after being previously suspended, when a user is recreated after being destroyed for spam or abuse, and when a network-wide suspension is in effect on an account.

³ A "destroyed" user is deleted along with all that they had posted: questions, answers, comments. Generally used as an expedient way of getting rid of spam.

⁴ This counts every review that was submitted (not skipped) - so the 2 suggested edits reviews needed to approve an edit would count as 2, the goal being to indicate the frequency of moderation actions. This also applies to flags, etc.

⁵ Includes close flags (but not close or reopen votes).

⁶ This ignores numerous deletions that happen automatically in response to some other action.

⁷ This includes comments deleted by their own authors (which also account for some number of handled comment flags).

Further reading:

Wishing you all a happy new year...

| |
  • 1
    I agree with Joe - will there be a post with stats on new questions, new answers and accepted answers; community reputation earned; number of questions/answers earning "good" and "great" badges; number of bad questions/answers; and other things about the community contribution beyond moderation. – HorusKol Jan 1 at 21:26
  • Most of that can be gleaned from public data; happy to assist there if you like @horuskol – Shog9 Jan 1 at 22:11
  • 2
    Question volume is visible in the analytics you can access with 10k reputation as part of the "moderator tools". Actual moderators have some additional views available but I'm not sure we're allowed to share extracts of those. – Lilienthal Jan 2 at 15:39
  • 2
    Oh well. I guess I don't understand the point of this posting. – Joe Strazzere Jan 2 at 20:38
  • 1
    @JoeStrazzere I believe the point is really just to provide an overview of moderation activities throughout the year. We occasionally have people wondering what kind of volume a site is dealing with and posts like these are the only way to see some of that information. But I agree that this is mainly just data, not information. We could do some kind of trend work by looking at previous threads or bringing in the query tool but ultimately there's not too much to be learned from doing that. Moderation volume scales with the site and each has its own ratios in the end. – Lilienthal Jan 3 at 0:36
  • @Lilienthal - I'm probably over-sensitive to this sort of thing. I worked for a boss who loved to publish lots of numbers just because they were easy. They provided no real value, and as far as I could tell nobody bothered to read them. Once he left the company, I got rid of most of the number, changed the reports to show trends, and provided commentary. It was a bit more work but was much better received. – Joe Strazzere Jan 3 at 0:40
  • 1
    @JoeStrazzere I understand where you're coming from. I agree that a lot more could be done if the community team wanted to invest some time in building reports but I can't speak for them. I assume the above is a pretty easy report to run and gives some basic numbers with little effort. Similar comments were made over on the main meta moderation thread. – Lilienthal Jan 3 at 0:45
  • Since by far the biggest activity involves comments, the obvious "management response" to this report should be simply to remove the option to post comments. Or if that is too draconian, auto-delete all comments after 24 hours. The second step would be get rid of moderators, since they wouldn't have enough work to do, and they already have far too much capability to create trouble (ref one well known recent example). – alephzero Jan 3 at 14:31
  • 2
    FWIW, @Joe: the origin of this report was something I used internally years ago. That followed with something I would share with mods occasionally, then something I'd post around controversial elections, and then now something I just post yearly because it's easier to point to a URL when someone has questions than it is to post this stuff in chat umpteen times throughout the year. I honestly don't know how many people find it interesting for its own sake; I kinda look at it like water co-op budget numbers: I don't care about them at all until they wanna raise my bill, then I wanna know why. – Shog9 Jan 5 at 2:15
  • @JoeStrazzere Did you ever go through ITIL training? Makes your point: Data is not information. Or as a mentor of mine liked to say "The map is not the territory" – Old_Lamplighter Jan 5 at 7:30
  • 1
    @RichardSaysReinstateMonica - No, I never went through ITIL training. I'm old. I worked in the business for many years. I've read many reports - some useful, some not. Back while the Earth was still cooling, I ran an IT department that produced a ton of printed reports. While reviewing the paper budget, I decided to run an experiment. I withheld one several-hundred-page paper report from all of the recipients (about 30). If they called asking for their report, I sent it. That way, I would learn who actually needed the report. Only 2 called. The report was a waste of time and money. – Joe Strazzere Jan 5 at 12:41
  • @JoeStrazzere I imagine you would enjoy this youtube.com/watch?v=BKorP55Aqvg – Old_Lamplighter Jan 5 at 14:49
12

I think this summary would be incomplete without mentioning the fact that in 2019 our site lost three out of five elected moderators.


Further reading for those interested in more details:

| |
1

I've been spending some time in the Stack Exchange Data Explorer trying to see if there are any ways to get data that may be more meaningful than the above, which doesn't really tell much of a story to me.

Of course, I do think it's up for debate how meaningful any of this is, but there were some specific figures which I found interesting. Lately, I've been spending a lot of time looking at review queues and I've noticed that we often have a lot of questions in the close vote queue. It doesn't seem like this queue gets ignored (there are often a lot of other people looking at it, compared to other queues) so that lead me to wonder if we simply vote to close really often. It seems like we do.

Per the above, we had 1,831 questions closed in 2019. I counted questions based on create date - we only had 4,149 questions asked in 2019. So we closed more than 44% of questions. To me, this means that either we are too aggressive with closing, or we don't go a good enough job of helping users understand what the close criteria are.

Of those 4,149 questions asked in 2019, 3,360 of them had at least one close vote. More than 80% of questions got a close vote.

If you sum the new questions by day and compare it to questions that received a close vote on that day, we frequently have days where the number of questions that received close votes is higher than the number of new questions asked that day. In other words, there are days where we are literally closing questions faster than they are being asked.

All of this lead me to wonder: What does this mean? It seems like we vote to close a lot but it's not immediately clear if that's problem or not. How do we know if it's a problem?

What are our goals as a community? I feel like I've picked up on a few common themes as answers to that questionn.

  • If we are here to curate a good body of knowledge (in the sense of maintaining a library of quality questions and answers), it seems like we are shutting people down faster than they can help us generate content. I didn't save the exact number, but something like 2,200 of our questions this year were from people who have only asked one question. We don't seem good at retaining askers.
  • If we are here to grow the user count as a way to create new users for SE at large, see above. We seem to be frightening away a majority of new people.
  • If we are here to help people who ask questions, closing 44% of their attempts to do so doesn't seem helpful.

Am I missing something? Do we have an opportunity to let this data tell us something about our behaviors, and how they relate to our goals? Do I have our goals wrong? Maybe this should be it's own question, but I posted here because the data posted above are what triggered my exploration.

| |
  • 1
    " It seems like we vote to close a lot but it's not immediately clear if that's problem or not. How do we know if it's a problem?" - I do think it's a problem. Anecdotally, we get a fair number of folks who seem unhappy with this "hostile" site. Analytically, it would be interesting to see how many first-time writers of questions never come back after having their sole question closed. – Joe Strazzere Feb 4 at 18:38
  • @JoeStrazzere that's a great next query to try when I get a few minutes, I'll update the answer then. – dwizum Feb 4 at 19:04
  • I had some queries open in a browser session that I lost before I could update this post, but new members who have asked more than one question have a slightly lower closed question rate than existing members, although new members with only one post have the highest rate overall (which I think supports your theory that a first question being closed is correlated to a member not returning for further activity). – dwizum Feb 6 at 18:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .