Instagram is Training AI to Detect Offensive Captions
Thu, September 29, 2022

Instagram is Training AI to Detect Offensive Captions

Since Instagram was launched in 2010, it has become a platform for a lot of people to communicate and share their lives with friends and even casual acquaintances / Photo by: ALEX_UGALEK via Shutterstock


Since Instagram was launched in 2010, it has become a platform for a lot of people to communicate and share their lives with friends and even casual acquaintances. Today, it boasts more than 800 million monthly active users, with more than 70 million photos uploaded on the platform every day. According to BBC, a British public service broadcaster, Instagram is the most popular social media platform among young people today, daily visited by 7% of the total population of the specific demographic. It is followed by Facebook (6%), Snapchat (5%), and Twitter and Youtube (2% each).People can interact with another there and show their love or support with a heart or comment, and the use of hashtags. Today, this social media is also using new technologies like artificial intelligence and machine learning to enhance its users’ experience. 

However, just like other online platforms, Instagram can become a toxic space for many. A 2017 study conducted by anti-bullying charity Ditch the Label revealed that Instagram is the number one social media platform for bullying as it has become a place full of mean comments.

Also, a survey of more than 10,000 young people showed that nearly 70% of youngsters admitted to being abusive online. About 17% claimed to have been bullied online. As a result, nearly 47% of the respondents stated that they don’t want to discuss bad things in their lives on social media. Many also preferred to show only an edited version of their lives. Ditch the Label's chief executive Liam Hackett stated that people augmenting their personalities online has become a trend lately.

“Cyberbullying continues to be one of the biggest challenges facing young people. Not only is the internet redefining the climate of bullying, but also it is having clear impacts upon the identity, behaviors, and personality of its young users,” Hackett said. 

Sheri Bauman, a counseling professor at the University of Arizona, who has spent years studying bullying’s causes and effects, called Instagram a “one-stop-shop for the bully.” Instagram head of public policy Karina Newton has recognized the fact that as their platform introduces new features, it opens new opportunities for abuse. Fortunately, AI can help to address this issue. 

Reducing Offensive Captions Through AI

AI is now being used in several functions. Most social media sites are using it to enhance their features as well as protect their users. Recently, Instagram announced that their AI system will start detecting and flagging potentially offensive captions that could be a sign of bullying. The system can detect if the caption a user posts has an offensive word in it. It will warn the user that the caption looks “similar to others that have been reported.” Also, it will give them the option to edit it. 

Scott Freeman, CEO of The Cybersmile Foundation, and Ross Ellis, founder and CEO of STOMP Out Bullying, believe this effort is a great way to limit cyberbullying on Instagram. Young people usually fear that their accounts would be suspended if they don’t follow the rules. Many of our youth could not exist without Instagram and wouldn’t take a chance to lose their account. But, keep in mind that these are cyberbullies. “If they are very technologically advanced they might try to find a workaround to the warning, move on to another social app, or bully the target in person,” Ellis said. 

Aside from reducing cyberbullying on the platform, this initiative also aims to educate people on what the company doesn’t allow on Instagram. While this is a huge step, Ellis believes that more should be done. This includes extending the effort on all of Facebook’s platforms. According to Forbes, a global media company focusing on business, investing, technology, entrepreneurship, leadership, and lifestyle, Liza Crenshaw, a Facebook company spokesperson, stated that they are planning to extend it to the platform soon. 

Instagram’s new initiative is part of the company’s long-term commitment to lead the fight against online bullying. It has been developing and testing AI that can recognize different forms of bullying on the platform. The feature will be rolling out in selected countries first before expanding in the coming months.

Censoring Hateful Comments

Instagram’s recent initiative to detect offensive captions is just one of its efforts to combat cyberbullying. Earlier this year, the company integrated a machine-learning algorithm to detect and block potential bullies on its platform. The AI-powered Rethink tool will flag comments that could be considered offensive before they are posted. Tests have shown that a notification like this has encouraged “some” users to “undo their comment and share something less hurtful.”

According to TechRepublic, an online trade publication and social community for IT professionals with advice on best practices and tools for the day-to-day needs of IT decision-makers, the platform is using DeepText, the same machine learning algorithm being used by Facebook to try and shut down their cyberbullying problem. This has given users a “chance to reflect.”

"It’s our responsibility to create a safe environment on Instagram. This has been an important priority for us for some time, and we are continuing to invest in better understanding and tackling this problem. I look forward to sharing more updates soon,” Adam Mosseri, head of Instagram, said. 

Most of the time, users are afraid to block a bully because it might lead to heavier repercussions. With this tool, users can block bullies without them ever knowing. Mosseri stated that they hope this effort can both prevent bullying and empower victims of bullying to stand up for themselves.

Above all, users must be educated to not use social media platforms as tools to bully someone. While many initiatives have been launched to combat cyberbullying, these are not enough until people learned that bullying is never a good behavior. 

Instagram’s recent initiative to detect offensive captions is just one of its efforts to combat cyberbullying / Photo by: asiandelight via Shutterstock