Stack Overflow bans ChatGPT temporarily

Today Stack Overflow moderators (myself included), have implemented a temporary ban on ChatGPT on the site.

Use of ChatGPT generated text for posts on Stack Overflow is temporarily banned.

This ban was picked up immediately by several technology news outlets, including ZDNet, TheVerge, and Vice. It was also picked up by CNN, The New York Times, and the Washington Post.

That's a lot of news coverage.

The question that I want to answer is "Why?". Stories and marketing from OpenAI, give reasons why this new chatbot is a "good thing". With their examples, it definately looks that way. But, in practice, it's not working out so well.

The responses that users have been posting on Stack Overflow have a high rate of being incorrect. Normally, the community can handle this, but the responses aren't your usual small code snippet of an answer. Instead it is a long, detailed, explaination that looks plausible. But, it's wrong. Combined with users posting multiple answers an hour, this is a lot of content that Stack Overflow reviewers (or worse, the handful of elected moderators) to go through and determine if it's valid.

At the time the ban was implemented, we'd seen thousands of answers generated by ChatGPT. On the one hand, this is impressive work on the ChatGPT AI itself. It can be difficult to detect and is good at holding a conversation. On the other hand, and more important from Stack Overflow's perspective, this isn't helping the user base. Thousands of subtly wrong answers is awful. It doesn't help the user looking for help, and it will very quickly destroy the trust that millions of developers put in the site if this is allowed to continue.

I'll admit that I'm disappointed that AI hasn't reached the point where it can do what ChatGPT seems to do. But, this is a step forward. Unfortunately, this step seems to have left a bad taste in the mouth of developers looking for help beyond a toy example.

For the time being, ChatGPT is banned on Stack Overflow. The moderation team will continue to work with the company to ensure the community we have volunteered to moderatate remains one of high quality. Additionally, as the larger Stack Exchange network of sites debates a similar ban, the Stack Overflow moderation team will be able to provide input on our experiences.