Pinterest uses machine learning to automatically scan billions of images and pieces of text across its platform for content that violates its policies, including adult content, hate speech, self-harm encouragement, medical misinformation, drug promotion, and graphic violence. The system works both in batches — reviewing all existing content daily — and in real time as new content is uploaded. Pinterest says reports of policy-violating content per impression dropped 52% after the system was first deployed in 2019.

The system uses neural network classifiers that take as input a combination of image embeddings (numerical representations of what an image looks like) and text extracted via optical character recognition (OCR). Two pipelines operate in parallel: a batch system that scans the entire Pin corpus on a regular schedule using distributed computing, and a streaming pipeline that flags new content within seconds of upload. As of late 2023, Pinterest had launched or updated 16 machine learning models covering categories including child safety, eating disorder promotion, depressive content, and weapons. The system handles tens of millions of deactivations per quarter — for example, 29.4 million Pins were deactivated for adult content in Q1 2023 alone.