Joining the Stack Exchange Moderator Strike

Summary¶

I'm signing an open letter to Stack Exchange because those of us that volunteer our time and energy have been put in an impossible situation. On Stack Overflow we banned ChatGPT created content almost immediately after it was released. Theorectically, that is not longer in effect.

This isn't due to a change in community perception. Instead, it's due to an abrupt policy change on Stack Exchange's part that was posted on May 30. It's important to note that this public policy does not match the guidance that was provided privately.

The relevant portion of this public policy is:

In order to help mitigate the issue, we've asked moderators to apply a very strict standard of evidence to determining whether a post is AI-authored when deciding to suspend a user.

The private policy notes that "very strict" is essentially "don't moderate unless they explicitly say it was created by an AI".

Background¶

The Stack Exchange CEO posted a blog post at the end of May 2023. In this post, they stated:

Approximately 10% of our company is working on features and applications leveraging GenAI that have the potential to increase engagement within our public community and add value to customers of our SaaS product, Stack Overflow for Teams.

This goes against the community desire to not have GenAI content on the site. The CEO has not provided any feedback to the community, other than a note that there is a big summer project with GenAI. The community reaction hasn't been positive.

Combining this announcement with the newly announced policy the previous day, and an astonishing inability to articulate more details or reasoning, has producted a lot of unhappy moderators.

Additionally, I feel that the company has destroyed 3 years of rebuilding trust. Back in 2019, they destroyed this trust and nearly had a moderator strike then too. That involved providing information to a journalist that had very limited context and due to that, presented a single moderator in unflattering light. The feelings from that have taken years to rebuild and even today that incident is cited as a low point, and users can point out that singular incident when trust of the company plummetted.

The public announcement to not moderate GenAI content contained this:

Through no fault of moderators' own, we also suspect that there have been biases for or against residents of specific countries as a potential result of the heuristics being applied to these posts.

That's just wrong. Absolutely wrong. It is 100% inaccurate and Stack Exchange has offered no data to back this up. User country of origin, region in the world, or any kind of physical location is not available to moderators. We are present a flag and given information to the content that has been flagged.

Why am I participating?¶

So...my participation? Why am I joining in?

Stack Exchange, and Stack Overflow, thrive due to the human element. It's userbase has been around over a decade and answered millions of questions across over 180 sites. The week ChatGPT came out, the community saw the bad results it can provide. For the past 6 months, we've continued to see how bad that is. The moderation teams across the network to generate a policy and get it approved by the company. I've reproduced it in full below, but as of this post it is still on the site and contridicts the new policy.

In addition to completely ignoring the community's input on how they do not want GenAI on the site, the company ignored their own moderator agreement.

With the assumption that the above will change at some point, the relevant section is pasted here:

``` Stack Exchange, Inc. agrees that it will:

i. Respect your privacy per the terms of the Privacy Policy for the Public Network. ii. Get your explicit written permission before commenting to any media (including media outlets controlled by Stack Exchange Inc.) or independent reporters about you or your moderator actions as per our Press Policy. iii. Allow you to resign your position for any reason without penalty or repercussions. As a volunteer, Stack Exchange, Inc. respects your time and will release you from duty should you ask. iv. Operate “Stack Gives Back”, an annual program giving to selected charities in honor of our moderators. v. Post previews for review of all new official policies in the Moderators Teams instance with the policy tag, marked with links to their public version once published, and maintain a listing of all official network-wide policies with links to them in the Help Center. vi. Announce changes to the moderator agreement no less than sixty days before the deadline to accept the new agreement with a period of at least thirty days for discussion and review. vii. Provide support for your questions, requests and concerns on the Moderators Teams instance and/or the Teachers’ Lounge, direct email to CMs, and content on Meta escalated to staff by whatever formal documented process is in effect at the time. viii. Respect your right to speak openly to question and challenge policy without reprisal so long as such speech does not break the Code of Conduct. ```

The relevant section is vi. There was no discussion period on this. No engagement with the moderators, or the community. Instead it was "effective immediately." The best we've gotten so far is an "oops, sorry."

That's not now something in a /legal link should operate. If I can't trust them to uphold an agreement they have in writing, why should I trust them to uphold anything else?

What does the community want out of this?¶

From the open letter:

Until Stack Overflow, Inc. retracts this policy change to a degree that addresses the concerns of the moderators, and allows moderators to effectively enforce established policies against AI-generated answers, we are calling for a general moderation strike, as a last-resort effort to protect the Stack Exchange platform and users from a total loss in value. We would also like to remind Stack Overflow, Inc. that a network that entirely relies on volunteers for its moderation model cannot then consistently ignore, mistreat, and malign those same volunteers.

My feelings on this¶

I've been on Stack Overflow and Stack Exchange for over 13 years. I've been a moderator on the network since 2014 and on Stack Overflow since 2017. I've written about Stack Overflow a bit over the years. I've participated in Charcoal, the spam fighting community since 2015ish. This project has automatically flagged more than 86,000 posts across the network since 2016.

I've applied to several jobs at Stack Exchange. I've interviewed for a couple positions. I have spent a lot of time with the network, the community, moderators, and employees making this a great place for internet users to find their answers.

I have helped to build the community/company trust that I mentioned above. I watched it crumble. I thought about leaving in 2019, but instead spent the next three years working to rebuild that trust. I'm at the point again, where I see the company not understanding their community. At all. It's sad that this cycle has repeated itself and it's worse that the company is, again, tossing their most engaged users under the bus.

This strike serves two purposes in my mind - the first is the officially stated one. Do not all GenAI content on the network. It will erode the value of the network quickly. We've also demonstrated that ChatGPT and it's peers are not great at answering complex questions. But, second, and unofficially, this strike will represent a change for Stack Exchange to show whether or not they care about what the community has to say. This is, I believe, the last opportunity for them to retract this policy and reflect on why they are over ruling so many communities that reject GenAI in their community. It's their last opportunity to show they support their moderators and the human aspect of moderation. Failure to do either means that Stack Exchange has given up on community building.

I am a volunteer for this community. I would love to continue that role, but this is the best way for me to show that GenAI is not the right path for the company to take.

What's Next¶

On June 5, 2023 at midnight, moderator local time, the network will start to see moderation activities cease (or slow down). I will be part of that.

GPT Policy¶

This policy was crafted with the input of moderators and Stack Exchange. An important thing to notice is that the moderators are empowered to issue suspensions. This is something the new policy prevents.

``` Why posting GPT and ChatGPT generated answers is not currently acceptable

This Help Center article provides insight and rationale on our policy regarding the usage of GPT and ChatGPT on Stack Overflow. While this is the position of Stack Overflow staff, it’s meant to support the prior work done by moderators (namely, the temporary policy issued to ban contributions by ChatGPT).

Stack Overflow is a community built upon trust. The community trusts that users are submitting answers that reflect what they actually know to be accurate and that they and their peers have the knowledge and skill set to verify and validate those answers. The system relies on users to verify and validate contributions by other users with the tools we offer, including responsible use of upvotes and downvotes. Currently, contributions generated by GPT most often do not meet these standards and therefore are not contributing to a trustworthy environment. This trust is broken when users copy and paste information into answers without validating that the answer provided by GPT is correct, ensuring that the sources used in the answer are properly cited (a service GPT does not provide), and verifying that the answer provided by GPT clearly and concisely answers the question asked.

The objective nature of the content on Stack Overflow means that if any part of an answer is wrong, then the answer is objectively wrong. In order for Stack Overflow to maintain a strong standard as a reliable source for correct and verified information, such answers must be edited or replaced. However, because GPT is good enough to convince users of the site that the answer holds merit, signals the community typically use to determine the legitimacy of their peers’ contributions frequently fail to detect severe issues with GPT-generated answers. As a result, information that is objectively wrong makes its way onto the site. In its current state, GPT risks breaking readers’ trust that our site provides answers written by subject-matter experts.

Moderators are empowered (at their discretion) to issue immediate suspensions of up to 30 days to users who are copying and pasting GPT content onto the site, with or without prior notice or warning. ```

The letter¶

The letter below was originally posted as an open letter to Stack Exchange. I've reposted it here for a record.

``` June 5, 2023 Stack Overflow, Inc. has decreed a near-total prohibition on moderating AI-generated content in the wake of a flood of such content being posted to and subsequently removed from the Stack Exchange network, tacitly allowing the proliferation of incorrect information ("hallucinations") and unfettered plagiarism on the Stack Exchange network. This poses a major threat to the integrity and trustworthiness of the platform and its content.

We, the undersigned, are volunteer moderators, contributors, and users of Stack Overflow and the Stack Exchange network. Effective immediately, we are enacting a general moderation strike on Stack Overflow and the Stack Exchange network, in protest of this and other recent and upcoming changes to policy and the platform that are being forced upon us by Stack Overflow, Inc.

Our efforts to effect change through proper channels have been ignored, and our concerns disregarded at every turn. Now, as a last resort, we are striking out of dedication to the platform that we have put over a decade of care and volunteer effort into. We deeply believe in the core mission of the Stack Exchange network: to provide a repository of high-quality information in the form of questions and answers, and the recent actions taken by Stack Overflow, Inc. are directly harmful to that goal.

Specifically, moderators are no longer allowed to remove AI-generated answers on the basis of being AI-generated, outside of exceedingly narrow circumstances. This results in effectively permitting nearly all AI-generated answers to be freely posted, regardless of established community consensus on such content.

In turn, this allows incorrect information (colloquially referred to as "hallucinations") and plagiarism to proliferate unchecked on the platform. This destroys trust in the platform, as Stack Overflow, Inc. has previously noted.

In addition, the details of the policies issued directly to moderators differ substantially from the guidelines outlined publicly, with moderators barred from publicly sharing the details.

These policies disregard the leeway historically granted to individual Stack Exchange communities to determine their policies, by making changes without the input of the community, overriding community consensus, and outright refusing to reconsider their position.

Until this matter is resolved satisfactorily, we will be pausing activities including, but not limited to:

Raising and handling flags.
Running SmokeDetector, the anti-spam bot.
Closing or voting to close posts.
Deleting or voting to delete posts.
Reviewing tasks in the various review queues.
Running various other bots designed to assist in moderation, such as detecting plagiarism, low-quality answers, and rude comments.

Until Stack Overflow, Inc. retracts this policy change to a degree that addresses the concerns of the moderators, and allows moderators to effectively enforce established policies against AI-generated answers, we are calling for a general moderation strike, as a last-resort effort to protect the Stack Exchange platform and users from a total loss in value. We would also like to remind Stack Overflow, Inc. that a network that entirely relies on volunteers for its moderation model cannot then consistently ignore, mistreat, and malign those same volunteers. ```