AI to spot violent and adult videos

02.05.2017 Interviews

Last week, after a tragic Thai video appeared on Facebook and Youtube, we received a request from Reuters Asia asking to briefly address some questions regarding the state of visual content filtering industry. A part of the interview was briefly mentioned in the Reuters’ article. In this blog post we are publishing a full version.

Reuters: What technology do you offer to automatically filter out undesirable content — violence, pornography? Could you provide a bit of background about how it works, and how it differs to other companies’ offerings?

Michael Pogrebnyak, Kuznech CEO: Detection of pornography is a significant subtask of content filtering problem on the internet. Studies have estimated that of the world’s 42 million websites, 12% contain adult content. Another statistical result estimated 70% of teens have unintentionally come across pornography on the web. This is, of course, alarming. The problem of detecting an adult-oriented content in images still remains unsolved and is widely addressed by many researchers. Over the past years there have been many approaches to detect and filter pornography.

Classification of porn video has been done by combining image features and motion information (Jansohn, Ulges, Breuel, 2008), repetitive motion detection (Endeshaw, Garcia, Jakobsson, 2008), skin detection (Stottinger, Hanbury, Liensberger, Khan, 2009), skin detection in an image to classify human body by applying geometric patterns (Fleck, Forsyth, Bregler, 1996), skin detection with using erotogenic-parts (Shen, Wei, Qian, 2007), features of previews (Rasheed, Shah, 2009), breast detection (Wang, Hu, Yao, 2009), complex method based on camera motion, skin detection and image features (Torres, 2012).

To contribute to solving the problem, Kuznech proposes an innovative approach based on logotype detection, text recognition and scene classification by convolutional neural networks (CNN).
One of possible ways to use our technology is to track illegal distribution of video content, too. Kuznech’s technology analyzes the uploaded videos and assigns to each file a special set of descriptors (hashes) to store them in the database. Each newly added video file is being assigned with new hashes. Algorithms compare descriptors of the input video with hashes of other videos in the database, reveal matching videos (those that have most hashes in common) and thus determines duplicates.

The goals of this application are:
— Filter and moderate user content uploaded to the website (tracks content delivered to the website (social network) and rejects prohibited visual materials (violence, pornography);
— Protect copyrights (the service tracks the usage of images and videos across databases of any size in order to prevent copyright infringement);
— Deduplicate the DBs of images/videos (organizes and clusters visual content to keep the libraries well managed and save hard drive space).

Reuters: What companies are using your software, and for what exactly? Are Facebook or Google among your customers?

Michael Pogrebnyak: We do not serve Google or Facebook, but among our customers there are notable clients, whose names we unfortunately cannot mention due to signed NDAs. Most common use case is when a huge social network with e.g. 10 000 000 video uploads/day demands some content filtering system to prevent prohibited content (violence, pornography) distribution over the network. It is also possible to detect such unwanted content in live stream videos (broadcasting).

Reuters: How do you see this technology and market evolving? And do you believe regulators will tighten requirements for social media companies in broadcasting videos?

Michael Pogrebnyak: At the beginning, pornography filters were mostly based on skin detection method. But this approach could not be used to detect violence, smoking, drinking etc.

As for future, over time the detection quality will grow (less FP <false positive> results will be in the results), and new, more complicated classes (categories) of unwanted content will evolve.
E.g. recently Yahoo open-sourced its porn detecting neural network, but there was no training set, only neural networks configuration. Without a training set, it would be very complicated to train the neural network to recognize violence – we assume, this can be done on videos, then.

In our opinion, it is absolutely crucial that the regulators oblige social networks (actually, all websites with huge volumes of UGC) to be supplied with such content filtering technologies for reasons of their users safety.

Reuters: In the Thai case, and last week’s murder in the U.S., do you believe your software would have captured the video before it was posted? What could have been done to prevent such cases?

Michael Pogrebnyak: This is possible, but it largely depends on the social network itself. If a the social network has installed a correct and qualitative content detection system, it will filter out the file on the step of its uploading into the network – and thus totally prevent its expansion to other users. The process looks like this: the uploaded video is analyzed frame by frame, and as soon as algorithms detect any “suspicious” objects (e.g. gun, blood etc.), the further uploading of this video gets blocked.