Instagram has revealed a new safety feature that uses AI technology to tackle cyberbullying. The addition of the Caption Warning setting uses AI to detect comments, phrases, and captions that have previously been flagged as harmful and alerts the user with an automated message advising them that the content they are about to post may be considered offensive.
After testing the tool earlier this year, Instagram found that the intervention encourages some people to undo their comment and share something less hurtful or offensive after being provided with an opportunity to reflect on their comment.
We should all consider the impact of our words, especially online, where comments can be easily misinterpreted. Tools like Instagram's Comment and Caption Warning are a useful way to encourage that behavior before something is posted, rather than relying on reactive action to remove a hurtful comment after it's been seen by others.
Dan Raisbeck, Co-founder, The Cybersmile Foundation
The new feature is one of the first to approach the problem of online bullying from a preventative perspective, allowing pause for reflection and time to reconsider content that could potentially harm or upset someone.
Instagram's AI can detect content that has been previously flagged as offensive, enabling it to respond to content that may not be recognized as harmful or offensive by users or moderators. The launch of this new tool is part of a wider effort by Instagram to address user safety on the platform.
Earlier this year, other safety features were introduced to protect Instagram users and ensure that time spent on the platform was safe and positive. These included a new 'Restrict' setting, which allowed users to block or filter out unwanted interactions without alerting the bully. This was well received by users who may have been reluctant to block or unfollow a bully through fear of retaliation or the situation escalating.