DR. FRED HEIDING

AI systems are rapidly lowering the cost and increasing the scale of phishing and social engineering attacks. After studying AI-enabled deception and fraud for over half a decade, we have compiled our knowledge into ScamBench (https://scambench.com/), a benchmark designed to systematically evaluate AI-enabled social engineering across email, voice, and multi-step attack scenarios, such as pig butchering.

This presentation describes how we model personalized attacks and measure their effectiveness against three datasets: human participants via real-world phishing simulations, synthetic users, and human participants via large-scale survey assessments. Each dataset measures three types of social engineering: personalized (based on information such as affiliations and collaborators), semi-personalized (based on information such as the user’s ZIP code), and generic. We present the methodology and results from each data stream, including our approaches to OSINT collection, email personalization, and the construction of individual vulnerability profiles. We also describe how we design synthetic users to closely mirror real-world behavior and demographics.

We then outline a range of defensive measures, including policy interventions such as strengthened KYC requirements for domain registrars, and practical guidance for individual users on data minimization (specifying which personal information to remove or retain based on its relative value to the user versus its utility to an attacker, as informed by our vulnerability profile analysis).

Finally, we discuss our ongoing collaboration with Frontier AI labs and how ScamBench could be used in pre-deployment security assessments to inform safer, more responsible model releases.