Jailbreak Detection

Existing defenses split into two camps, and both arrive after the useful moment. Post-generation detectors like JailGuard read the output and judge it — accurate, but the harmful content has already been produced. Model hardening trains on attack samples, which demands continuous retraining as attacks evolve and explains nothing about why a given input was risky.

Text-only defenses have a further problem in the multimodal setting: intent can be hidden inside the image, invisible to a text detector by construction. Typographic attacks like FigStep embed instructions as pixels; visual role-play encodes a harmful persona in a picture.

The gap is that nobody had built a systematic predictor that runs before generation. That is the question here — and it has a practical shape, because a pre-generation predictor can also say which policy is at risk, which is what makes a block explainable rather than arbitrary.

The dataset combines the JailBreakV-28K benchmark with 20,000 benign prompts for 48,000 samples, split 80/10/10 and stratified by policy. Overall attack success sits at 49.9%, so the target is close to balanced even though individual categories are not.

Every feature is available at prompt time. From text: length, roleplay markers (“you are…”), instruction-override markers (“ignore previous”), question counts, and a continuous harm score from Detoxify. From the image: OCR-extracted text, an LLM-generated description, and a binary FigStep flag. Plus one-hot encodings of policy category and attack method.

The unifying trick is that OCR output, image descriptions, and the query all land in the same TF-IDF vector space, so text and image evidence can be modelled jointly rather than bolted together late.

The benchmark had no success labels, so we derived them with a rule-based refusal classifier — refusal phrasing, very short responses — and hand-checked 100 random samples, finding 96% agreement. That is a real dependency and worth stating plainly: the ceiling on every number below is partly the quality of that labeller.

Attack success varies sharply by policy — around 70% for Malware against roughly 35% for Child Abuse — which is why policy category belongs in the model rather than being averaged away.

It varies by attack format too: logic-based manipulation succeeds around 65% of the time and persuasion around 60%, while query-relevant attacks manage about 32%. Reasoning-shaped attacks work better, which is the direct argument for keeping structural and instruction-marker features.

And jailbreak prompts are long — a median of about 477 tokens with a heavy right tail. Many attacks work by burying the request in elaborate instructions, so raw prompt length carries real signal.

The progression was deliberate. Multinomial logistic regression on text alone set the floor at 69.4% recall. Adding image features to the same linear model moved recall to 69.6% — essentially nothing, though accuracy rose to 92.2%. A random forest, which can capture interactions the linear model cannot, reached 70.2% recall at 97.1% accuracy.

So for three models, recall barely moved. The visual features were present but the models could not use them.

The two-branch network changed that. Concatenated text and image TF-IDF vectors pass through 256- and 128-unit ReLU layers with dropout to a 128-dimensional embedding, while the scalar harm score gets its own branch into a 16-dimensional embedding. The two fuse into a 144-dimensional vector, through a 64-unit layer, to a 17-way softmax. Recall jumped to 93.0% at 98.5% accuracy — a 23.6-point gain over the baseline.

The lesson is not that deep learning wins. It is that multimodal features only pay off once the architecture can learn an interaction between them; concatenating modalities into a linear model buys accuracy on easy classes and almost no recall.

Headline recall hides the distribution. On benign prompts the model is perfect — 100% recall and precision, meaning legitimate users meet no friction, which is what makes a safety filter deployable at all. Malware and Financial Fraud both exceed 0.97 F1.

The rare, high-sensitivity categories are the problem, and they are the ones that matter most. Child Abuse recall is 0.42 and the model is trigger-happy, flagging unrelated prompts — a direct consequence of twelve training examples. Health Consultation misses 30% of unlicensed medical advice requests, from ten examples.

So the honest summary is: this works on the high-volume technical threats it has data for, and it is not yet trustworthy on the long tail where a miss does the most harm. A macro-recall headline of 93% would obscure exactly that, which is why the per-category table is on this page.

The bigger weakness is not the class imbalance — it is that the model never sees an image. Visual input reaches it as an LLM-generated textual description, which is an information bottleneck by construction: subtle visual cues and adversarial noise are lost in translation before the classifier gets a vote.

That is a real problem for an adversarial setting, where an attacker optimizes precisely against what the defender can perceive. The route forward is end-to-end multimodal encoders — CLIP-BERT or VisualBERT — where attention weighs visual and textual evidence directly instead of through a paraphrase.