I’m a Master’s student at Kookmin University in Seoul, working on AI security, AI safety, and the policy questions between them. I write explanations of attacks and defenses and write-ups of my own research, with the code to rerun them.
Writing
OWASP’s AI security guide, applied to an email agent
OWASP’s AI Exchange explains prompt-injection defense with an email summarizer; run on the one I built, its questions find the real design flaw in a few steps, but the evidence behind its specific advice is thin.
Prompt-injection detectors, tested on prompts they should let through
A detector with F1 0.988 still blocked about a third of harmless, security-related prompts from public datasets. No threshold fixed it, and extra training only fixed the prompts we had written ourselves.
Indirect prompt injection, tested on an email agent
An email agent with a mock send tool and a fake secret, 360 poisoned inboxes, two OpenAI models, no leaks, and why that is less reassuring than it sounds.
Prompt injection, explained by running one
A twenty-line summarizer, a document with two extra sentences, four local models, and why this is a harder problem than SQL injection ever was.
Paper notes
WildGuard: the refusal label was the right call, the GPT-4 comparison is weaker than it looks
WildGuard was the first open moderation model to label prompt harm, response harm and refusal separately, and that part holds up. Its margins over GPT-4 sit inside base-model noise, GPT-4 helped grade the test set, and later work shows a refusal sentence in front of a harmful reply makes the guard call it safe.
Measuring the Wrong Thing: the jailbreaks that work score lower than the ones that fail
On Llama-3.1-8B, a harmfulness score that separates harmful from harmless prompts well ranks the jailbreaks that succeed below the ones that fail; the broader claims rest on thinner evidence.
XSTest: the over-refusal test holds up, its automatic grader less so
XSTest’s human-labelled results reproduce exactly from the released files, but the GPT-4 grader the paper offers as a cheaper substitute agrees with those labels far less than its tables suggest.
PIGuard and NotInject: what the over-defense result shows, and what it doesn’t
NotInject shows public prompt-injection guards blocking many harmless prompts that contain attack-like words, but PIGuard’s own gain is hard to read, because its released code picks the model using items from the test sets.
Publications
- PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift DOI · arXiv · Benchmark and code