The study conducted by OpenWeb and Perspective API analyzed 400,000 comments that some 50,000 users were preparing to post on sites like AOL, Salon, Newsweek, RT and Sky Sports.
Some of these users received a feedback message or nudge from a machine learning algorithm to the effect that the text they were preparing to post might be insulting, or against the rules for the forum they were using. Instead of rejecting comments it found to be suspect, the moderation algorithm then invited their authors to reformulate what they had written.
"Let's keep the conversation civil. Please remove any inappropriate language from your comment," was a message prompt or "Some members of the community may find your comment offensive. Try again?"
In response to this kind of feedback, a third of internet users (34%) immediately modified their comments, while 36% went ahead and posted their comments anyway, taking the risk that they might be rejected by the moderating algorithm. Even more surprisingly, some users made modifications that did not necessarily make their comments kinder or less hostile.