|Facebook uses a machine learning framework called deep entity classification (DEC) to detect fake accounts / Credits: metamorworks via Shutterstock|
Facebook has been removing fake accounts on their platform for years. Originally, a team would identify a set of features and label each as “abusive” or “benign.” Some of the features include an account’s age, number of friends, and location, which they use to train an account classifier model. However, the feature space was relatively small, which makes it easier for attackers to understand. Thus, Facebook uses a machine learning framework called deep entity classification (DEC).
DEC was developed to address problems Facebook has been encountering in its traditional approaches to automated fake account detection. According to VentureBeat, an American technology website that publishes news, analysis, long-form features, interviews, and videos, the framework is responsible for a 20% reduction in abusive accounts on the platform in the two years since it was deployed. This is equivalent to hundreds of millions of accounts.
The recent Community Standard Enforcement Report revealed that Facebook was able to remove more than 3.2 billion fake accounts between April and September 2019. This is much higher compared to 1.5 billion during the same period last year. DEC was able to detect these accounts because it used a multi-stage, multi-task learning technique using large amounts of low-precision, automatically generated labels in tandem with small amounts of high-precision human-provided labels.
When detecting a fake account, DEC considers an account’s direct features by entity type first. This includes age and gender (user entities), fan count and category (page), member count (group), operating system (device), and country and reputation (IP address). After that, it expands to other entities the account interacts with, like pages, admins, group members, users sharing a device, groups shared to and registered accounts.
“Over the past few years that DEC has been in production, we’ve seen a step reduction in the number of [abusive] accounts on the platform. Even though attacker volumes increase, DEC catches them at pretty much the same volume,” Facebook software engineer Sara Khodeir said.