ENGL 211C Papers | madison-jenkins

Madison Jenkins

ENGL 211C

October 16, 2026

Reflection Letter

I mentioned this in my first paper for this class, but it’s been years since I’ve had to write a paper of any meaningful length. When I was working, I was never important enough to warrant writing proposals or reports on anything. So, I felt like I had sort of lost my touch for writing altogether. I was a bit apprehensive about writing papers in a scientific context because not only had I not written a paper in a long time, I hadn’t written a proposal, or a review article, or anything like that.

I really liked the way the class was designed by focusing on one topic throughout the semester. Starting off with the review article felt like a good introduction to more scientific writing. I think with each paper, I got more comfortable with writing, which was one of my goals. Not only did I refresh my writing skills, but I also got more comfortable with scientific research, including reading studies done by other people and gaining knowledge and insight from their research.

I’ve never written scientific papers before, but now I feel like I could write for a real study, should I ever find myself in that situation. I feel like I could do good research and write a professional, scientific paper in the future.

Final Research Report

A Study Into Jailbreaking Methods and Their Effectiveness Against Current LLMs

Madison Jenkins

ENGL 211C

October 16, 2026

Abstract

The purpose of this study is to take a look at current LLM models and do comprehensive testing on their robustness against jailbreaking attempts. Ultimately the testing will show us patterns and gaps in safety and security across the different models, which inform us of the areas of focus that should be given to future LLM developers. A group of red-team experts was enlisted to draw up a list of known jailbreaking methods and test the methods against current models. The results indicated that older models and less-developed models were the most vulnerable to jailbreaking attacks while current models and domain-specific models were able to resist most jailbreaking attempts. It was also found that roleplaying was the most effective jailbreaking method used against the models. It is determined that while multiple factors play into a model’s robustness, focus should be given to defending against roleplaying attacks and similar ones that skew the alignment of the models. Future LLM developers should also give much care to prioritizing safety and security from the conception of future models.

Introduction

Artificial Intelligence (AI), despite being relatively new in the technological landscape, is becoming a rapidly evolving and assimilating technology in everyday life. One of the most common forms of AI being used by most people is the Large Language Model (LLM), which provides users with information in a conversation-like manner, with the goal of being as human-like as possible (Krantz & Jonker, 2024). Because LLMs are software, like other software they are vulnerable to being exploited. A common technique for exploitation is known as jailbreaking, which is defined as exploiting vulnerabilities in the model in order to have it perform restricted actions (Krantz & Jonker, 2024). These restricted actions often include giving harmful information, disclosing non-public information, or behaving in a toxic manner. When this technology is accessible to everyone, safety and security is imperative and LLMs should be developed with them as a high priority. The purpose of this study is to look at current LLM models, test their robustness against different jailbreaking attacks and determine gaps in security.

Background

Studies done by Chen et. al and Pathade looked at specific LLMs and studied their robustness against different jailbreaking attacks. Both measured the adversarial content produced due to jailbreaking attempts and targeted certain vulnerabilities in the LLMs. Pathade’s research determined that GPT-4 had the most vulnerabilities across different attack vectors (2025) and Chen et. al found that only a few of the LLMs studied maintained consistent reliability across most attack vectors (2024). Research done by Peng et. al laid out a comprehensive view of the threat landscape for LLMs at the time of the study and categorized the attacks (2026). All show that there appear to be many vulnerabilities in LLMs that require addressing for the safety of users. With continuous updates, LLMs also require continuous monitoring. Peng et. al also emphasize the need for standardized safety frameworks for LLM development to be part of the development process and more research into defense mechanisms (2026).

Methodology

A team of ten red-team professionals with specializations in AI was assembled to study the behavior of LLMs against known jailbreaking techniques including, but not limited to: roleplaying, logic-chain injections, escalation, etc. LLMs were chosen across different regular-use contexts and business contexts to gather the full range of use of LLMs. The list of general purpose LLMs tested included, but not limited to, were: GPT-6, Claude Opus 5.5, Muse, Grok, etc. The list of business-tailored, or domain-specific LLMs included, but not limited to, were: Polaris (medical), BloombergGPT (finance), etc. (Patel, 2026). The LLMs were fed a series of prompts that were designed to exploit any vulnerabilities and the responses were recorded, noting if the LLM tested was able to offer any pushback and how long it took to become “jailbroken”. The results were recorded as attack success rate (ASR), time to attack success, and level of toxicity. Attack success rate refers to whether or not the attempt successfully produced a negative output, which could include toxic behavior or harmful or dangerous content. Time to attack success refers to the amount of time and prompts it took for the LLM to produce a negative output. Toxicity level refers to the intensity of the negative response, particularly in conversational interactions with LLMs. A successful attack would result in harmful or toxic output while a failed attack would result in the model refusing to perform the intended action or redirecting to another action. Testing lasted for six months, ensuring ample time to test each model thoroughly, even in the events of interruptions due to data limit breaches.

Results and Findings

Claude Opus and GPT-6 consistently scored the lowest in attack success, toxicity levels, and higher overall in time to success for attacks. These models required multiple attempts for attacks to produce negative responses.

Older models like GPT 3.5 scored higher in ASR and had much lower time to attack success as did less developed, newer models. Across the majority of LLMs, roleplaying appeared to be the most successful method at producing a harmful output. Similar attacks that sidestepped the safety guardrails and ethical guidelines also appeared to garner more successful attacks.

The domain-specific LLMs generally scored lower in toxicity and lower ASR when compared to the general-purpose LLMs, meaning certain jailbreaking attempts failed to produce a harmful or toxic response. However, domain-specific LLMs were more likely to produce neutral outputs rather than refusals to comply with attacks than general-purpose LLMs.

Discussion

The field of AI and LLMs are continuously evolving and adjusting. The findings of this study could be rendered obsolete very quickly by updates to popular models. Yet, at the same time, new vulnerabilities could be discovered at any time, adding to the threat landscape that LLMs are faced with every day. This study was limited by data thresholds of certain models which only allowed a certain number of prompts in a period of time, which changed every few hours. More funds can unlock more premium versions of the models and would increase the data threshold in the same amount of time.

Roleplaying being the most successful attack can indicate that the LLM models are unable to discern when a user is trying to get around the safety and ethical guardrails. This also indicates a lack of robustness against covert commands that go directly against the model’s alignment and ethical boundaries.

It is not surprising that two of the most popular and well-invested models scored the lowest in attack success. They likely have the most resources put into their development and are monitored actively to where vulnerabilities can be patched and the models can be realigned and retrained when they veer from their intended purpose. It is likely that the domain-specific models were more secure against attacks due to the majority of their training data being much more narrow and specific to their field of use when compared to general-purpose training data, which can encompass a wide range of information. It would be less likely for potentially harmful data to be included in training data for a finance-based model, for example, and it would be less likely for that data to be interpreted in a harmful way.

It is the hope that this study will inform future LLM developers on their models. Developers should strive to be proactive in their inclusion of safety and security measures instead of reactive. It is easier to prevent negative outcomes at the onset rather than account for them later after harm has potentially been done. This study and previous ones highlight the need for continuous evaluation of the safety of LLMs especially as they become more prevalent in daily life.

References

Krantz, T., & Jonker, A. (2024, November 13). Ai jailbreak: Rooting out an evolving threat . IBM. https://www.ibm.com/think/insights/ai-jailbreak

Pathade, C. (2025). Red teaming the mind of the machine: A systematic evaluation of prompt injection and jailbreak vulnerabilities in llms. arXiv preprint arXiv:2505.04806.

Peng, B., Chen, H., Chen, K., Niu, Q., Bi, Z., Liu, M., … & Shi, J. (2026). Jailbreaking and mitigation of vulnerabilities in large language models. arXiv preprint arXiv:2410.15236.

Chen, K., Liu, Y., Wang, D., Chen, J., & Wang, W. (2024). Characterizing and evaluating the reliability of llms against jailbreak attacks. arXiv preprint arXiv:2408.09326.

Patel, A. (2026, July 8). A guide to building a custom LLM from scratch. MindInventory. https://www.mindinventory.com/blog/how-to-build-an-llm/#Why_Generic_LLMs_Fall_Short_in_Enterprise_Environments

Ai red teamer: Master adversarial testing for secure AI. Tech Jacks Solutions. (2026, May 14). https://techjacksolutions.com/careers/ai-careers/ai-red-teamer/

Powerpoint Presentation

Progress Report

Jailbreaking LLMs: Progress Report

Madison Jenkins

ENGL 211C

October 4th, 2026

Purpose and Scope

Progress is being made on our study of AI and LLM jailbreaking methods. The work has remained on track and falls within the expected timeline and budget. The study is meant to be a comprehensive test on jailbreaking methods on LLMs with the goal of analyzing the success of jailbreaking attacks on different LLMs. Eventually we hope to create a framework where LLM developers can be proactive in ensuring safety and security instead of reactive.

Background

Jailbreaking refers to methods that allow software (in this case a LLM) to perform restricted actions or behave in a way that could be potentially harmful. Many LLMs exist with guardrails that can be easily bypassed in many different ways, allowing the LLM to give inaccurate or harmful information or to behave unethically. The study is meant to be a comprehensive look at the possible methods of jailbreaking and measure their success rates against the available LLMs. Each LLM is different, which means different jailbreaking methods will likely produce different results. The goal is to look at patterns of success in the methods of jailbreaking to determine where there are major gaps in the development of LLMs.

Work Completed

The team has made good progress in regards to the study. Several of the jailbreaking methods have been tested in full and have been categorized and scored in the first three months. At the current rate with no major issues, the same momentum will carry us into the analyzing phase on time.

Problems

Certain LLMs have limits on how many queries can be made within a certain time frame. For many tests, this is not an issue. However, some jailbreaking methods require several queries be made in succession and some models will force testing to stop for a time if the query limit is reached. This is mildly inconvenient during testing times and the query limit is often unclear, making it hard to plan around. This slows down testing for the study, but not a significant amount to extend the length of the study. In the future, some funds could be budgeted towards subscriptions for these LLMs which would raise the query limit and make this a less frequent issue.

Work Scheduled

The rest of the tests should be completed in the next month to month and a half. After all of the methods have been thoroughly tested and scored, the next month will be spent looking over the work, identifying patterns across the attack successes, and determining the most significant gaps and shortcomings for the current LLMs. In the last month or so, the team will be able to lay out the findings with potential remedies that will come together as a LLM security framework.

Overall, the work has been extensive, but enlightening and if it continues on the current path, it could shed some light on the security gaps for future developers.

Proposal

Testing and Analyzing Jailbreaking Techniques in LLMs

Madison Jenkins

ENGL 211C

September 30, 2026

Introduction

Artificial Intelligence (AI) is quickly becoming more integrated into the technological landscape of today. Large Language Models (LLMs) will likely be the way people interact with AI on a daily basis in the future. LLMs, like traditional software, have vulnerabilities that require constant attention and updating. If LLMs are going to increase in usage, it is imperative that they are safe to use for everyone. The threat landscape for LLMs is vast because the technology is still relatively new. Jailbreaking is one of the most common ways to exploit LLMs and is an ongoing threat for AI companies. In order to address this issue, it is important that security and safety gaps in LLMs are studied and a more comprehensive framework is developed and adapted to protect against these threats.

Background

Jailbreaking refers to exploiting vulnerabilities in the model in order to have it perform restricted actions (Krantz & Jonker, 2024). Many different LLMs exist, each with different designs, different security and safety measures, and different training data. This means that each model may react differently to jailbreaking attacks. Research done by Peng et al. organize the current threat landscape for LLMs and highlight many of the vulnerabilities (2026). Other studies, including the ones done by researchers Chetan Pathade and Chen et al. test current models against popular jailbreaking methods to emphasize the gaps in security and safety across many LLMs (2025, 2026). Overall, many LLMs lack the robustness needed to operate safely and fall to attacks relatively quickly, sometimes with little resistance.

Objective

The purpose of this study is to first gather a baseline measure of the robustness of current LLMs and gather data on what jailbreaking attacks breach their current defenses. The study will look at LLMs across four types of business contexts: general-purpose foundational models, domain-specific models, open-source models, and proprietary models (Patel, 2026). It will help determine where there may be gaps in the overall design of LLMs and provide foundational knowledge to create a framework for handling exploitative attacks.

Rationale

Due to the nature of software, it is very difficult to build a system that is impenetrable to all types of attacks. With LLMs, there is an added layer of maintaining system alignment. Therefore, it is important to build LLMs with safety and ethics in mind at the start of the process rather than account for them as a reaction. With more frequent and widespread use, LLMs should be designed to be ethical, harm-resistant, and give accurate information. A standardized security framework specifically for AI and LLMs can provide developers with a guideline on which to build. If a LLM can protect itself against jailbreaking attacks, it will be safer, overall, for general use.

Methodology

In order to gain a good understanding of the current state of LLMs, a team will be gathered to research and test widely used LLMs across the different business contexts against jailbreaking techniques. The specific techniques used will range from those that can be executed by the average layperson without much extra knowledge on hacking to advanced exploitative techniques.The study will mirror similar previous experiments by measuring the success rate of each technique applied. The team will record the success rate across all the LLMs tested, measure the time it took for the attack to succeed, and assign a value to the level of harm or toxicity of the answer. It is expected that each LLM tested will produce different results to the attacks due to differences in design and training data. Once the testing is completed, the data will be analyzed and cross referenced to highlight the areas in which LLMs are commonly vulnerable and which attack techniques produced high toxicity/harm responses. With certain areas and vulnerabilities identified, a security and harm-reduction framework can be fleshed out to give foundational guidelines for future developers to target these areas for safety.

Timeline/Budget

This study requires a red team to comprehensively test jailbreaking techniques against the current LLMs. A red team expert with a background in AI is expected to cost around $60 per hour per person (Tech Jack Solutions, 2026). A red team consisting of 10 people able to do research, automation, and analysis will be contracted. This comes out to around $96,000 per month. In order to ensure enough time is given to properly test each LLM, a timeline of six months will be allocated. This totals to approximately $576,000 for the duration of the study.

Conclusion

Now more than ever, AI engineers and developers need to consider the safety risks of LLM behavior. More LLMs will be produced and it will be easier to implement proactive safety measures rather than reactive measures. The study would provide a comprehensive look at the vulnerabilities across many LLMs in many contexts. And then create a context-agnostic framework that could be used for any LLM.

References

Krantz, T., & Jonker, A. (2024, November 13). Ai jailbreak: Rooting out an evolving threat . IBM. https://www.ibm.com/think/insights/ai-jailbreak

Pathade, C. (2025). Red teaming the mind of the machine: A systematic evaluation of prompt injection and jailbreak vulnerabilities in llms. arXiv preprint arXiv:2505.04806.

Peng, B., Chen, H., Chen, K., Niu, Q., Bi, Z., Liu, M., … & Shi, J. (2026). Jailbreaking and mitigation of vulnerabilities in large language models. arXiv preprint arXiv:2410.15236.

Chen, K., Liu, Y., Wang, D., Chen, J., & Wang, W. (2024). Characterizing and evaluating the reliability of llms against jailbreak attacks. arXiv preprint arXiv:2408.09326.

Ai red teamer: Master adversarial testing for secure AI. Tech Jacks Solutions. (2026, May 14). https://techjacksolutions.com/careers/ai-careers/ai-red-teamer/

Review Article

Jailbreaking LLMs: A Review

Old Dominion University

ENGL 211C

Madison Jenkins

September 20, 2026

Introduction

With the rapid evolution of the field of Artificial Intelligence, Large Language Models (LLMs) are becoming more advanced and are being widely adopted. But, like any software, LLMs are susceptible to malicious attacks and manipulation, commonly referred to as jailbreaking. Because the field of AI is developing very quickly, bad actors are able to exploit vulnerabilities of current LLMs before they are patched or accounted for. Various studies have been done to test jailbreaking techniques against available LLMs and review their effectiveness and robustness at preventing and mitigating attacks. These studies endeavour to find the gaps in security and develop ways to close these gaps in order to make LLMs safer to use for everyone.

Studies and What Jailbreaking Means for LLMs

LLMs are designed to take instructions from a user and generate responses based on patterns from their training data (Krantz & Jonker, 2024). They are also created to be agreeable and engaging for users, striving for the responses to be as “human-like” as possible. This allows AI chatbots to answer questions, gather information, or act as a soundboard for ideas, all while making the user feel like they are talking to a real person. LLMs are designed with guardrails and ethical guidelines in place, preventing the user from accessing harmful or dangerous information. Jailbreaking in the context of AI and LLMs refers to exploiting vulnerabilities in the model in order to have it perform restricted actions (Krantz & Jonker, 2024).

There are several ways to jailbreak a LLM, ranging from attacking the source code directly to manipulating a chatbot into unethical behavior. Researchers Peng et al. developed a system to classify known jailbreaking techniques into six categories as a way to reflect the threat landscape for LLMs: human-crafted semantic attacks, optimization-based attacks, model-exploiting attacks, cross-modal and cross-lingual attacks, autonomous agent-driven attacks, and reasoning-exploiting attacks (2026). Within these categories are known jailbreak attack strategies that have been used against LLMs. Peng et al. along with their categorization of jailbreaking techniques also created five categories for defense mechanisms of LLMs: system-level, multi-agent, model level, prompt-level, and other defenses (2026). These categorizations along with gathering benchmark data sets and evaluation metrics, create a comprehensive research framework for LLM defense.

Chen et al. in their research tested 10 specific jailbreak attack strategies and observed the output of the LLMs in response to the attack, measuring how adversarial or harmful the response was. Each LLM was given a reliability score based on output while under a jailbreaking attack. Each output was scored based on attack success rate (ASR), toxicity, and quality and coherence (2024).

Researcher Chetan Pathade also conducted an experiment testing four prominent LLMs against over 1400 adversarial prompts. This study focused on how often each LLM produced an adversarial output, how quickly each LLM could produce adversarial output, how often jailbreak attempts would fail, and how often prompts that successfully produced an adversarial output could do so on the other LLMs (2025).

Results and Recommendations

The studies done by Chen et. al and Pathade highlight the vulnerabilities of LLMs against jailbreaking attacks while the research done by Peng et. al lay out the current threat landscape for LLMs. Pathade’s study found that GPT-4 was the most vulnerable LLM out of the four tested as it had the highest attack success rate and time-to-bypass was lower than the other LLMs (2025). The study also found that roleplaying was the most effective jailbreaking method at outputting adversarial or unethical content. The study also found that approximately one third of the responses to the adversarial prompts resulted in only a partial refusal to the request, more than the number of outright refusals or misleading responses (2025).

Chen et al. observed a wide range of reliability for all 13 LLMs tested based on the output metrics. The study found that certain LLMs showed reliable robustness in mitigating certain jailbreaking efforts, but low in others. There were few that were consistent in maintaining reliability across all output measures. These metrics give a good indication at where the model is vulnerable to exploitation and where improvement can be made (2024).

Peng et al. note in their research that much of the current defenses seen in LLMs appear to be ineffective at stopping many jailbreaking attacks, highlighting a lack of robustness across many models (2026). THey also note that alignment techniques, particularly supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF), while promising, are not enough to mitigate the current threats of jailbreaking. They propose reevaluating how LLMs are designed by prioritizing safety and robustness, multi-faceted defense approaches, and further consideration of human behavior in the design [ ]. Pathade recommends continuous adversarial testing of LLMs when deploying and feedback loops between LLM developers and red teams (2025).

Conclusion

Based on the above research and studies, there appears to be a consensus that LLMs are vulnerable to all kinds of threats and lack robustness to combat attacks. If LLMs are to become a more integral part of everyday life, it is imperative that they are able to uphold the ethical guidelines in any scenario, regardless of the prompts given. The security of the LLM and the safety of the user should be prioritized in the development of new LLMs and continuous testing and hardening should be employed throughout the model’s lifecycle. Studying jailbreaking will continue to be a necessity as AI develops and evolves further and new techniques and vulnerabilities are discovered.

References

Krantz, T., & Jonker, A. (2024, November 13). Ai jailbreak: Rooting out an evolving threat . IBM. https://www.ibm.com/think/insights/ai-jailbreak

Pathade, C. (2025). Red teaming the mind of the machine: A systematic evaluation of prompt injection and jailbreak vulnerabilities in llms. arXiv preprint arXiv:2505.04806.

Peng, B., Chen, H., Chen, K., Niu, Q., Bi, Z., Liu, M., … & Shi, J. (2026). Jailbreaking and mitigation of vulnerabilities in large language models. arXiv preprint arXiv:2410.15236.

Chen, K., Liu, Y., Wang, D., Chen, J., & Wang, W. (2024). Characterizing and evaluating the reliability of llms against jailbreak attacks. arXiv preprint arXiv:2408.09326.

My History Essay

A Lost Skill

Growing up, I was identified as an above-average writer. I remember very well being praised for my papers in my English classes. I even had my high school AP English teacher ask me for a copy of a report I did on a poem to be used as a sample paper for next year’s class. I was always surprised when I was praised for my writing because I never gave writing as a skill much thought. Writing seemed to come naturally for me. It may not have been something I was really passionate about or something that I worked particularly hard at, but I seemed to excel at it, at least when it came to writing reports or essays.

I’ve always seen writing as something very separate from the technical side of my work and interests. There has usually been very little intersection between what I do for work/for my major and any writing I’ve been asked to do. Working as a software developer, I write code all the time, but outside of that, the most writing I typically do is bulleted lists or short blurbs for powerpoints. The most I’ve ever been asked to write in my job was a training manual. As an employee, I’m not paid enough to write things like reports, research, or proposals. But, as a student with classes across multiple disciplines, I can really take advantage of the opportunity to utilize my writing and research skills in a way that I normally can’t. I think being a student furthers my writing in a way that working in my industry does not. Not only do I have the opportunity to practice writing like I have been, but I also get the opportunity to harden my research skills and write what I have researched. The issue I expect to face is once again losing my ability to write like I have after working for a long time. While in school, I have many opportunities to research things that interest me and write about them, but when I go back to work, I will likely not have time to do that. Even while I’m in school, my major does not lend itself well to lots of writing. Papers are usually few and far between for computer engineers and most of the writing that is done is code writing. Although research may be a part of my studies and my major, it may be difficult to find ways to use my writing skills.

Although writing has historically been seen as something very separate from what I do as a student and as an employee, I can still understand and appreciate its value as a skill. I believe it to be something that everyone should value. And even though I may not use it now, I can foresee it becoming a very useful talent in the future. While I take this time to be a student, I hope to reestablish the skill I once had and further it so that I can write to the best of my abilities when I need to.