Gray Swan AI
Gray Swan AI builds Cygnal for automated red-teaming and Shade for jailbreak defence. Founded by Carnegie Mellon researchers, it works with frontier labs.
Highlights
- Frontier AI safety company building Cygnal (red-teaming) and Shade (jailbreak defenses)
- Founded in 2024 by Zico Kolter and Matt Fredrikson (Carnegie Mellon CS faculty) plus Andy Zou
- Researchers behind some of the most-cited adversarial-ML and jailbreak papers
- Approximately $5.5M+ seed; backed by Pillar VC and others
- Partners with OpenAI, Anthropic, and other frontier labs on red-teaming
- Runs public jailbreak competitions that expose frontier model weaknesses
- Used by AI deployers and regulators to evaluate safety before production rollout
External link — opens grayswan.ai in a new tab. Gray Swan AI is a third-party product; we are not affiliated with it.
Link checked 17 November 2026: this site responded and still names the product.
About Gray Swan AI
What it is
Gray Swan AI builds Cygnal for automated red-teaming and Shade for jailbreak defence, founded by Carnegie Mellon researchers and working with frontier labs. It also runs public red-teaming competitions, which is where much of its attack knowledge comes from.
Why it's different
Its credibility comes from the research rather than the marketing: the founders published significant work on adversarial attacks against language models, and working with the labs means exposure to attacks that succeed rather than ones in a benchmark. The competitions are a clever mechanism for staying current, since a bounty attracts people who find what automated probes miss. The honest framing is that no defence here is complete — jailbreaking is an open research problem and defences are broken by new techniques regularly. Treat a guard as a substantial risk reduction, never a guarantee.
How people use it
It is used by organisations deploying models where a harmful or manipulated output would be a serious incident, and by teams needing evidence of adversarial testing for a regulator or customer. The valuable output is the specific attacks that succeeded against your system, not the aggregate score — those tell you where your particular deployment is weak, which is the only thing you can act on.
Written by the n3os team. We are not affiliated with Gray Swan AI.
More Coding & Development tools
- Adaptive SecurityPaid
Adaptive Security simulates AI-powered social engineering, such as deepfake voice calls and SMS phishing, against a company's staff, then trains them.
- Future AGIFreemium
Future AGI is a platform for finding out whether an LLM application works, bundling simulation, evaluation, tracing and guardrails. It is self-hostable.
- Glow SecurityPaid
Glow Security is an endpoint platform that inventories and controls every piece of software running across a fleet, not only what an agent looks for.
- Horizon3.aiPaid
Horizon3.ai runs NodeZero, an engine that continuously executes real attack techniques against production environments to find exploitable weaknesses.
- Prophet SecurityPaid
Prophet Security automates the investigation stage of security operations: taking alerts, gathering context and presenting analysts with a conclusion.
- StarseerPaid
Starseer is an AI security and interpretability platform. It scans models you did not train for vulnerabilities, then monitors activations at runtime.
Every Coding & Development tool in the directory →
Listed by how many tags they share with Gray Swan AI, then by name. Nobody pays to appear here.
This listing was written from public information, without Gray Swan AI’s involvement. If you own it and something here is wrong — or you would rather not be listed at all — email us and, once we have confirmed it is yours, we will correct or remove it.
Get the ones worth knowing about
We write one of these for every tool worth the trouble. Get the new ones, plus what we have found genuinely useful lately.
Your address goes to Buttondown, who send the emails on our behalf. One click unsubscribes, and the list is never sold or shared.