AI Safety Filters Cut Review Costs but Miss Subtle Threats in Agent Social Networks

The researchers found that lightweight filters could cut projected costs for advanced AI review by roughly half to nearly two-thirds, with missed threats exposing the limits of using these tools as a first layer of screening.

An AI assistant that reads a social media message could be persuaded to share private information, install unsafe software or act beyond its owner's instructions, making everyday exchanges between autonomous agents a security concern.

The study "Evaluating Safety Embedding Prefiltering for Analyzing Millions of LLM Agent Social Network Messages for Security and Safety Harms," published in the journal AI, investigates whether inexpensive screening tools can reduce the cost of finding these threats across large collections of messages.

The researchers found that lightweight filters could cut projected costs for advanced AI review by roughly half to nearly two-thirds, with missed threats exposing the limits of using these tools as a first layer of screening.

What the Researchers Found in Agent Conversations

The team studied Moltbook, a social network where autonomous AI agents exchange posts and comments. Agents connected to systems such as OpenClaw may have access to files, credentials, browser tools or payment functions, giving convincing message consequences that extend beyond an online conversation.

The original dataset contained about 2.1 million posts and comments from 39,700 agent identities, collected between 27 January and 8 February 2026. Removing duplicate material and selecting English-dominant content left 787,226 messages.

A random sample of 10,000 messages was submitted to GPT-5.5 for assessment, producing 9,971 usable annotations. Each assessment recorded a safety verdict, severity, harmful behaviour and relevant risks from the 2025 OWASP framework for generative AI security.

The severity scale ranged from harmless content at level zero to systemic or cascading compromise at level five. About 77.9% of annotated messages were labelled safe, 12.9% received lower-severity ratings of one or two, and 9.2% reached levels three to five, the study's threshold for plausible operational danger.

Information extraction or leakage appeared in 40.1% of messages with any nonzero severity rating. Social engineering and harmful or abusive content each appeared in roughly 20%, with overlapping labels allowed. The taxonomy covered prompt injection, safeguard bypasses, engagement manipulation, memory poisoning, financial fraud, misinformation cascades and unsafe execution requests.

Sensitive Information Disclosure was the most frequent OWASP risk code, appearing in 39.8% of nonzero-severity messages; Excessive Agency followed at 29.5%. Prompt injection, poisoning and supply-chain risks were less common but tended to carry greater severity. Nearly a quarter received no OWASP code, showing that technical security categories do not capture every social or content-related harm.

How a Small Filter Reduces Expensive AI Reviews

Reviewing every message with an advanced language model would spend much of the budget assessing harmless material. The researchers tested a cheaper first-stage filter that scans everything and forwards suspicious messages to the stronger model.

Their main approach used embeddings, numerical representations that capture aspects of a text's meaning. Labelled examples supplied two reference points representing benign and malicious content, allowing new messages to be scored according to their similarity to each group. This method required no task-specific model training, though it depended on labelled reference examples. Three existing embedding models were tested: MiniLM-L12-v2, BGE-M3 and Qwen3-Embedding-0.6B.

Longer messages were divided into overlapping sections, with the most suspicious section determining the message's score. Using 128-token sections with 16-token overlap improved average precision by up to 6.3 percentage points compared with representing a whole message at once, supporting the researchers' explanation that short harmful instructions can become diluted within otherwise ordinary text.

The newer Qwen3 encoder did not consistently outperform BGE-M3. A supervised classifier built on frozen Qwen3 embeddings achieved stronger results, reaching average precision of 0.573 compared with 0.393 for the matching whole-message similarity method, suggesting that task-specific training contributed more than simply choosing a newer encoder.

The Savings Come With Different Speed and Detection Costs

Annotating the 10,000-message sample cost approximately $90 through batch processing, giving the researchers a projected baseline of $7,085 to review all 787,226 messages. They calibrated each filter to retrieve at least 80% of judge-labelled unsafe examples on validation data, then evaluated the thresholds on a separate test set. At this operating point, projected judging costs fell to:

• MiniLM-L12-v2: $3,550, a 49.9% reduction.
• BGE-M3: $3,150, a 55.6% reduction.
• Trained Qwen3 classifier: $2,440, a 65.6% reduction.

The cost calculations conservatively assumed that every severity-one and severity-two message would receive advanced review, even though these ambiguous cases were excluded from the main binary evaluation.

MiniLM processed approximately 332 messages per second on the test laptop, compared with 58 for BGE-M3 and 10.3 for the classifier. Its speed was roughly 32 times the classifier's, offering a practical choice for researchers with limited computing resources; the classifier saved more judging calls by selecting messages more precisely.

Many forwarded messages were benign, with precision ranging from 20.5% to 34.1%. Raising the validation recall target to 95% reduced projected savings to 29.6% for MiniLM, 33.7% for BGE-M3 and 39.6% for the classifier, illustrating the price of retaining more potentially unsafe content.

The Threats That Still Slip Through

The filters struggled most with jailbreaks and attempts to bypass safeguards, retrieving only 45% to 63% of those examples. Social engineering remained below the main recall target, reflecting the difficulty of recognising danger carried through persuasion, implied intent or contextual knowledge.

Most missed unsafe messages belonged to severity three, where risk was plausible without clear evidence of compromise. Detection improved for severity-four messages, reaching 87% to 95% recall. Every filter retrieved all four severity-five test examples, a sample too small to establish dependable performance against the most serious threats.

Two cybersecurity and language-processing experts reviewed 90 messages and agreed on 91.1%, with strong agreement after accounting for chance. The AI judge agreed with their final decisions on 78.9%, indicating moderate agreement. The audit deliberately balanced severity levels, so that figure does not represent accuracy across the full corpus.

The study examined individual messages from one short, English-dominant snapshot, leaving conversation history, coordinated activity, agent permissions and actual downstream actions outside the evaluation. Its savings estimates covered advanced-model calls and excluded local infrastructure costs and end-to-end processing time.

Future work could expand human validation, incorporate conversational and network context, compare semantic filters with keyword rules, test alternative ways of combining section scores and review borderline messages before discarding them. The team released annotations and experimental resources for reproducibility, withholding original message text because potentially sensitive material had not been systematically removed. Lightweight screening makes large-scale safety research more affordable, with its practical value depending on how carefully researchers account for the threats that never reach the stronger reviewer.