GuidesAI and agents

Moderating content with AI: reviews, comments, listings

By the Banzai Labs teamUpdated on 3 min read

PublishedTo reviewRejectedClear cases decided automatically, edge cases seen by a human
Clear content is published or rejected automatically; edge cases go to human review.

Three outcomes

  1. 1PublishedWithin the rules, beyond doubt.
  2. 2To reviewIrony, edge case, doubt: a human decides.
  3. 3RejectedClear violation, reason recorded, author informed.

Rules, written down

Category Example
Insults, harassment Attacking a person
Personal data A third party’s phone number or address
Spam, advertising Link to another shop
Off topic A review that isn’t about the product

The model receives these rules and returns, for each piece of content, a category, an outcome and a one-sentence justification.

Setting the thresholds

Where to set the slider
Strict
  • More content in review
  • Fewer mistakes published
  • For a sensitive audience
Lenient
  • Less review
  • A few mistakes removed afterwards
  • For high volume

Follow-up

  • A sample of automatic decisions reviewed every week.
  • Appeals counted: a rising rate signals a badly written rule.
  • Rules versioned, so you know which version decided.

For structured output: reliable JSON from an LLM. The legal frame: AI, GDPR and the AI Act.

Frequently asked questions

Can the AI reject on its own?
For obvious cases (insults, spam, personal details), yes, with a recorded reason. Edge cases stay human.
Should the author be told?
Yes. Saying why content was rejected and how to appeal avoids frustration. In Europe, the Digital Services Act (DSA) points the same way for hosting services.
Are negative reviews rejected?
No. A negative but respectful review is published. Filtering it distorts the reviews, and may be illegal.

Read next

All guides