Content Warning: This post and, in particular, the contents of the sources for this post, contain references to, and images of, graphic violence and abusive comments.
On Sunday, the Guardian released a series of excerpts from Facebook’s moderation guidelines . These guidelines are for use by moderators to determine which content should be taken down, censored to certain age groups, or marked as disturbing ensuring that content does not automatically play on a user’s timeline. The guidelines have caused some controversy, which the Guardian appears to be revelling in, and Yvette Cooper has made a public statement about the inadequacy of the Facebook moderation process . With good reason; Facebook live has enabled the streaming of a host of disturbing content as of late including videos of murder and sexual assault .
Before we judge Facebook’s moderation rules, it is worth looking at how they are enforced and the problems Facebook faces when trying to do so. Facebook uses a number of tools to identify content that needs to be removed, the simplest being asking users to flag content. You can flag any content as spam, offensive or inappropriate and that content will be removed from your timeline (you can also remove something from your timeline without reporting it) . This feature alone is not sufficient to moderate content, however – it took over an hour for the Facebook Live video of the murder of Robert Godwin Sr. to be reported . In order to tackle this problem, Facebook is exploring new AI techniques which can identify, for example, sexual imagery so that they can be automatically reported .
Another issue with the current system is that users may be reporting content which does not actually contravene Facebook’s community standards . As such, Facebook employs around 1,000 moderators worldwide to check flagged material . They have the ability to delete content, disable user accounts or pass content on so that it can be reported to the police . Given that a moderator’s job essentially consists of looking at disturbing and graphic images all day, and that the position is quite poorly paid, the majority of moderators quit between 3 and 6 months of starting and many likely suffer from PTSD as a result of their employment . The task of moderators is enormous – there are millions of content reports to sift through a week . As such, Facebook is hiring 3,000 new members of staff to help with the moderation , however the efficacy is always going to dependent on their training and the internal guidelines provided by Facebook.
So, what do the guidelines actually say? The Guardian has leaked extracts from documents on sextortion and revenge porn, sexual activity, sex and nudity in art, non-sexual child abuse, credible threats of violence, graphic violence and cruelty to animals . These documents outline how to determine whether or not content should be removed, marked as disturbing, escalated or ignored. Just before we look at the guidelines in more detail, I think it’s worth noting that, because of the way that the Guardian has chosen to present these files, it is not possible to evaluate their completeness. Potentially, this is just a small extract of the documentation provided to moderators, or maybe this is all they get. It is also unclear from this leak whether moderators receive additional training and guidance. The Guardian has stated that they have removed some distressing content from the Facebook Files, but haven’t been clear about how much (they claim to have removed ‘many’, that well known standard unit of measurement). As such, it’s only really possible to evaluate the material available and recognise that this is not the same as fully evaluating Facebook’s moderation process.
Having said that, the Facebook Files seem to suggest a pretty woeful moderation policy. The guidelines appear to be explained through example and it is often not clear whether these examples are a complete set or not. For instance, the globally protected vulnerable groups listed in the Credible Guideline Policy are homeless people, foreigners, and Zionists . (This means that violent threats are only automatically considered credible if they are directed at these groups and not to any others.) There are no clear rules on why some groups are considered vulnerable and some aren’t and the documentation does not explicitly say whether or not these are the only vulnerable groups, or not. Assuming that this documentation is all moderators have to go on, they are likely very unprepared for content which is not included in the documentation, or for content that cannot be clearly linked to an example given.
Moderation policies built on example, as opposed to clear definitions, are poor because they leave a lot of space for individual moderators to decide the parameters of what is acceptable content beyond the examples they are given. This allows for the policy to be enforced unequally (a flaw in and of itself) and inadequately. If Facebook has a specific idea of who they consider to be vulnerable, for example ‘activists’, they need to clearly state it and not give a handful of examples (tbf, they haven’t even done that) and allow moderators to give their best guess.
Facebook’s policy appears to be centred around keeping as much content up as possible. They will allow content that calls for violence against certain groups or individuals as long as that content wouldn’t actually increase that person/group’s chance of facing violence: “we aim to allow as much speech as possible but draw the line at content that could credibly cause real world harm” . I’m not going to pass judgement on the approach Facebook is taking to decide what should and should not stay up (just yet). But, I would like to point out that “real-world harm” is a pretty subjective concept and therefore a shaky foundation upon which to build your moderation policy. Do you consider someone feeling threatened or upset to be a real-world harm? If so, a non-credible threat may still contravene your policy. Once again, the guidelines allow for individual moderators to decide for themselves what content is and is not ok.
Some of the moderation policy is actually built around hard and fast rules, presumably to limit this subjectivity. For example, in their guidelines on sexual content, the guidelines are explicit in that “general threats of exposure” do not need to be removed, whereas specific ones should be, with some (more) examples to illustrate the difference between the two . It seems to be that where rules can be applied, they are provided. Whilst it is frustrating that there are undefined terms in the documents, the terms left undefined are actually quite hard to define (and they may well be given greater context in other areas of the guidelines). It is also worth noting that explicit rule based systems can cause their own problems in the sense that they can be gamed. For example, if you know that Facebook won’t consider threats credible if they contain the phrase “when hell freezes over” instead of a real-world date or time, you might start using that phrase to threaten people. A moderation system becomes particularly vulnerable when someone leaks all of these rules to your users, see the Good Fight episode 6 (Social Media and its Discontents) for an example. (As adorable as I think the show and its attempts to discuss online issues are, I don’t think its portrayals are realistic, just fyi.)
One section of the guidelines which I actually think is quite well-handled is that on depicting graphic violence. Facebook both wants to allow users to share graphic images of important events, for example to raise awareness of or provoke discussions on conflict, and to stop users from posting images of violence that they condone and enjoy viewing. It is genuinely difficult to draw the line between these two groups – do you consider the photograph of Alan Kurdi’s body to be a powerful image that has helped to humanise the plight of refugees fleeing the Syrian conflict or an image of dead child taken without the consent of their parents and shared despite the child’s father’s express wishes? Instead, Facebook has chosen to determine whether or not content should be removed not by looking at the content itself but by looking at the context in which it was shared. If a video or picture sharing graphic violence is done so in a way that “express[es] sadism” it is removed . And this time, sadism is actually defined (they dedicate 4 whole slides to its definition) demonstrating employees at Facebook can actually use a dictionary.
Ultimately, it seems, Facebook is aiming for a moderation policy that allows for the most content possible and so are only willing to remove content is it is likely to put other people off from participating on the site and producing their own content. This seems fair, given the nature of their business, even if it seems distasteful or bizarre, for example because their default position is to allow images of animal cruelty (it is hard to not be judge-y) .
Now, I don’t want you to think that I like Facebook from this post. Some of their approaches are pretty naïve, for instance, their policy to leave up content (or, as they put it, “evidence”) of child abuse to help identify children . Just take it down and pass it onto the authorities. I also think that individuals depicted in any content shared on Facebook should have the right to demand it be taken down, even if that content does not otherwise contravene the moderation policy. But, other than those two (really, quite straight forward and obvious things), I certainly think that Facebook has a better moderation policy than the Guardian headlines would lead you to believe.
- https://www.theguardian.com/news/2017/may/22/no-grey-areas-experts-urge-facebook-to-change-moderation-policies?utm_source=dlvr.it HYPERLINK