Mohammad Asfour and Juan Carlos Murillo of Cornell University conducted a case study to identify how simulated human targets respond to social engineering attacks, specifically phishing. This was done with the goal of getting a better understanding of the human factors that influence the success rate of social engineering attacks.

The case study used OpenAI’s Playground, with the GPT-4 model, to create 20 distinctive agents (the “humans”) each based on a distinctive quality associated with one of the Big Five personality traits. Each of these was attacked three times with the same phishing email and the responses were collected to identify which of the Big Five were most susceptible to social engineering attacks.

Of the 20 attacks, only eight agents were successfully manipulated. In addition to the strict pass/fail criteria, the researchers also collected the written email responses of the agents for further analysis. This study has very good potential to assist society in training development, a key concern to any CISO, and opening the lens for other researchers to use Large Language Models to advance the social and cyber sciences.

The data was collected and presented in a straightforward manner, and in line with the principles of science. The researchers asked a very small question in a very big topic, allowing easy application of objectivity, parsimony,  skepticism, and empiricism. The small question additionally allowed the general public and their peers to digest and comprehend the goals, process, and outcomes of their case study.

Asfour, M., & Murillo, J. C. (2023, August 30). Harnessing Large Language Models to Simulate Realistic Human Responses … Harnessing Large Language Models to Simulate Realistic Human Responses to Social Engineering Attacks: A Case Study. https://vc.bridgew.edu/cgi/viewcontent.cgi?article=1172&context=ijcic