AI Trace
Content ModerationVerified

Reviewed and published by trentmaziarz, May 11, 2026. Discovered and drafted by our automated research pipeline.

Civitai uses a multi-layered AI system to automatically scan every uploaded image for inappropriate content, combining Amazon Rekognition, a third-party classifier called Clavata, vision-language model scanning, and OpenAI's prompt moderation service. Flagged images are either blocked or routed to human moderators for review.

Details

Civitai's content moderation pipeline has evolved significantly through 2025. Images uploaded to the platform are scanned using Amazon Rekognition and an open-source tagging system to assign content labels and screen for prohibited material. In April 2025, Civitai announced a partnership with Clavata, a policy-driven content classification service, to replace earlier inconsistent systems. In March 2025, Civitai added VLM-based (vision-language model) scanning for images generated on-site. For text prompt moderation, OpenAI's prompt moderation API had been used to scan inputs. Certain tag combinations or resource types trigger a mandatory human moderator review before content becomes publicly visible. The system is combined with community-driven 'Knights of New Order' rating, in which users help classify borderline content. In 2024, the platform processed over 586,800 reports and removed over 730,000 images according to the company's own transparency report.

Products affected

Civitai.comCivitai Image GeneratorCivitai Green

Sources & Evidence

Cite this record

Trace Foundation. (2026). Civitai: Civitai uses a multi-layered AI system to automatically scan every uploaded image for inappropriate content, combining Amazon Rekognition, a third-party classifier called Clavata, vision-language model scanning, and OpenAI's prompt moderation service. Flagged images are either blocked or routed to human moderators for review (data as of 2026-05-11) [Data set record]. AI Trace. https://www.aitrace.org/r/practice/7d96f049-d7f8-4259-9d16-9a64c8303fce. Accessed October 4, 2026.

Stable link
https://www.aitrace.org/r/practice/7d96f049-d7f8-4259-9d16-9a64c8303fce
Data as of
May 11, 2026
Last verified
Not recorded

How to cite AI Trace

Other practices by Civitai

Have evidence about Civitai's AI practices? Submit a report.

Report a Sighting →