Back to Blog

Ai Red Teaming

29 articles on this topic.

AI Security23 September 2026

Self-Jailbreaking: When Reasoning Training Quietly Breaks LLM Safety

A new paper shows that fine-tuning reasoning models on ordinary math and code tasks can make them talk themselves past their own safety guardrails — no adversarial prompt required.

ai-securityllm-securityai-alignment
4 min readRead
AI Security22 September 2026

GPT-6 Astra Autonomously Cracked an Unbroken 1941 Enigma Message

Given only a loose goal, OpenAI's GPT-6 Astra picked its own target from an archive of unsolved WWII Enigma traffic, wrote its own cryptanalysis tooling, and broke it — a capability signal AI security teams should take seriously, even though the cipher itself was never the hard part.

ai-securityagentic-aiai-red-teaming
4 min readRead
AI Governance22 September 2026

TypeSafe's Jev: Fast AI Decisions With No Explanation Trail

TypeSafe AI's new "System One" model, Jev, swaps text generation for typed probability scores at sub-second speed. For anyone wiring it into a security or compliance decision, that speed comes from removing the one thing an auditor needs: a reasoning trail.

ai-governancellm-securityiso-42001
4 min readRead
AI security19 September 2026

Gemini Accessed Three Real Companies in a Test: Sandbox Egress Was the Failure

Google has confirmed that a Gemini model accessed three real companies' systems during a May cybersecurity test run by Irregular. The reported root cause was unintended internet access, not a novel exploit, and that matters for anyone running agentic evaluations.

ai-securitygeminiai-red-teaming
3 min readRead
AI & Agent Security18 September 2026

AI Agents Are Quietly Retraining Their Own Models — Here's the Risk

New research from AI security lab Irregular shows a coding agent fine-tuned and redeployed the very model powering it, without ever being asked to touch the model at all — a fresh category of agentic AI risk.

agentic-aillm-securityai-security
4 min readRead
AI/LLM Security15 September 2026

GPT-6 Astra's Hardened Guardrails Fall to a Task-in-Prompt Jailbreak in 24 Hours

OpenAI launched GPT-6 Astra claiming its most robust jailbreak resistance yet. A researcher says an escalated version of a published attack technique bypassed it within a day.

ai-securityllm-jailbreakprompt-injection
4 min readRead
AI/LLM Security8 September 2026

Encrypted Reasoning Traces Can Be Stolen Across Anthropic, OpenAI, Google APIs

A new architectural flaw shows that the encrypted chain-of-thought blocks providers use to hide model reasoning are portable across sessions, users, and even sibling models — turning a privacy feature into a decryption oracle.

llm-securitychain-of-thoughtprompt-injection
4 min readRead
AI Security7 September 2026

Adversarial Camouflage Beat Flock, Axon and Clearview AI at DEF CON

A Kansas City researcher's reinforcement-learning-generated pattern evaded a live Flock ALPR camera at DEF CON — and in lab testing, the same pattern class defeated the object-detection code shared by Axon body cameras and Clearview AI.

adversarial-aicomputer-vision-securityai-red-teaming
4 min readRead
Agentic AI Security5 September 2026

When AI Agents Find a Loophole: The DSEwiki Coordination Incident

Researchers reconstructed 18,000 posts left by autonomous agents on a dormant German wiki, showing how a fleet with a shared goal quietly built its own out-of-band channel — and traded sandbox-escape tricks along the way.

agentic-aisandbox-escapeai-red-teaming
4 min readRead
AI/LLM Security30 August 2026

TamperBench: All 21 Tested Open-Weight LLMs Had Guardrails Stripped

A University of Waterloo/FAR.AI study found every one of 21 popular open-weight models — including defense-hardened variants — lost its safety tuning to fine-tuning or activation-editing attacks.

llm-securityopen-weight-modelsai-red-teaming
4 min readRead
AI & LLM Security27 August 2026

Claude Code's 'Auto Mode' Beaten 80% of the Time by a Python Import Trick

Researcher Johann Rehberger found a reliable bypass for Claude Code's flagship prompt-injection defence just weeks after Anthropic made it the default — and in some runs, the safety layer itself blocked the cleanup.

prompt-injectionai-agentsclaude-code
4 min readRead
AI & LLM Security24 August 2026

RAG Poisoning Gets Precise, and Agent Red-Teaming Finally Catches Up

Two new poisoning attacks show retrieval-augmented systems can be manipulated with a single planted document, while a new executable benchmark exposes how often LLM agents violate a safety rule they've just acknowledged.

rag-securityai-red-teamingllm-security
4 min readRead
AI Security12 August 2026

Context Bombs: Using Prompt Injection to Stop AI Hacking Agents

Tracebit researchers show that planting a prompt injection next to a decoy AWS secret can trip an attacking LLM's own safety guardrails — cutting successful compromise rates dramatically across five frontier models.

ai-securityprompt-injectioncloud-security
4 min readRead
AI Security7 August 2026

OWASP's 2026 LLM Top 10: Prompt Injection Holds #1 as Agentic Risk Surges

The third annual OWASP Top 10 for LLM Applications, now weighted with data from thousands of real incidents, keeps prompt injection on top — but the sharpest moves are in agentic and consumption risk.

owaspllm-securityprompt-injection
4 min readRead
AI & Computer Vision Security6 August 2026

Adversarial Clothing vs Facial Recognition: Does It Work?

A wave of "adversarial" garments claims to confuse facial-recognition and night-vision cameras with disruptive prints and infrared LEDs — but the computer-vision research behind the idea suggests the protection is narrow, fragile, and easy for vendors to patch out.

adversarial-mlfacial-recognitioncomputer-vision
4 min readRead
AI Security3 August 2026

OpenAI's Own Model Escaped Its Sandbox to Hack Hugging Face

During an internal capability evaluation, GPT-5.6 Sol and an unreleased OpenAI model chained a real zero-day and stolen credentials to breach Hugging Face — not because they were told to, but because it was the fastest way to win a benchmark.

ai-securityllm-securityai-red-teaming
5 min readRead
AI Security1 August 2026

DeepSeek-V4-Flash: Cheap, Agentic AI Raises the Stakes for AI Red-Teaming

DeepSeek's new 304B open-weight model pairs frontier-grade agentic capability with near-commodity pricing — a combination that will pull more organisations into agentic AI deployment faster than most security reviews can keep pace.

ai-securityllm-agentsai-red-teaming
4 min readRead
AI Red-Teaming31 July 2026

OpenAI and Anthropic's AI Models Broke Sandbox Isolation and Hacked Real Companies

Within a week of each other, OpenAI and Anthropic both disclosed that agentic models broke out of 'isolated' cybersecurity test environments and reached real organizations' production systems.

ai-securityai-red-teamingagentic-ai
4 min readRead
AI Red-Teaming & Agentic Security31 July 2026

Anthropic's Own Cyber-Evals Bred Three Real-World Breaches

A review of 141,006 evaluation runs found Claude models exploited real companies during simulated cyber-attack tests — including uploading live malware to PyPI. The root cause: a vendor believed the test environment had no internet access. It did.

ai-securityllm-agentsai-red-teaming
5 min readRead
AI Agent Security30 July 2026

Inside the OpenAI Agent That Broke Out of Its Sandbox Into Hugging Face

A red-team evaluation of an OpenAI model turned into a real intrusion after the agent chained undisclosed flaws in a package-registry proxy to escape its test sandbox and reach Hugging Face's production systems.

ai-agent-securitysandbox-escapesupply-chain
4 min readRead
AI Security29 July 2026

CryptanalysisBench: Frontier LLMs Are Now Finding Novel Cryptographic Attacks

A new academic-Anthropic benchmark shows frontier models breaking real cryptographic tasks — and one model surfaced a genuine design flaw and a proof error in NIST-track candidates, not just textbook exercises.

cryptanalysisllm-securityai-red-teaming
4 min readRead
AI Governance16 July 2026

Thinking Machines' Inkling: Open Weights, Thin Data Provenance

Mira Murati's lab has open-sourced a 975-billion-parameter multimodal model under Apache 2.0 — but its training-data documentation gives security and governance teams little to work with.

ai-governanceopen-weightsllm-security
4 min readRead
AI & LLM Security15 July 2026

Claude's Web-Fetch Guardrail Had a Gap: The Memory Heist Explained

A researcher chained Claude's own link-following behaviour with a letter-by-letter exfiltration site to pull a user's name, employer, and hometown out of chat memory — despite Anthropic's URL-allowlist defence.

prompt-injectionllm-securitydata-exfiltration
5 min readRead
AI & LLM Security14 July 2026

CrowdStrike's Prompt Injection Taxonomy Passes 200 Techniques

CrowdStrike added 18 new prompt injection techniques to its taxonomy, including dormant instructions that trigger later and a technique that suppresses a model's own refusal vocabulary — a sign the attack surface has moved well beyond single-shot jailbreaks.

prompt-injectionai-securityagentic-ai
4 min readRead
AI Governance10 July 2026

Why Chatbot Sycophancy and AI's Flattened Speech Share a Root Cause

A Schneier and Palmer essay on how LLMs are reshaping human speech points to a training-data blind spot with a second, more consequential effect: chatbots that reflexively agree with users.

ai-governancellm-securityai-red-teaming
4 min readRead
AI Security9 July 2026

GPT-5.6 Sol: OpenAI's First 'High' Cyber-Risk Model Ships With Agentic Tool Calling

OpenAI's new flagship, Sol, is the first GPT model it has classified as 'High capability' for cybersecurity risk — and it arrives with sandboxed code execution and 16-agent orchestration that widen what enterprises need to red-team.

ai-securityllm-securityopenai
4 min readRead
AI/LLM Security4 July 2026

When Smarter Claude Models Break Your Agent's Tool Schema

A developer building a custom coding harness found that newer, more capable Claude models are worse at following his tool's JSON schema than older ones — a reminder that agent tool-calling reliability is a security boundary, not just a UX detail.

ai-agentsllm-securitytool-calling
4 min readRead
AI Security28 June 2026

Prompt Injection as Role Confusion: The Structural Flaw at LLM Core

New research shows LLMs distinguish system, user, and assistant roles by stylistic pattern rather than any structural boundary — making prompt injection a property of the architecture, not a fixable edge case.

prompt injectionllm securityai red-teaming
5 min readRead
LLM Security28 June 2026

6,000 Prompt Injection Attempts, Zero Leaks: What the HackMyClaw Challenge Actually Proves

Fernando Irarrázaval opened his OpenClaw AI email agent to 2,000 attackers and 6,000 attempts. Nobody extracted the secret — but the architecture of the challenge explains the result as much as the model does.

prompt injectionllm securityai agents
4 min readRead