Ahmed Kargbo
Cyber Ethics
Dr. Linford
Novel Cybersecurity Risks
Case Study 5
Upon reading and analyzing the three ChatGPT articles from Dropboxtech, CPO magazine, and Github, I will be focusing on the issue of training data extraction attacks which is also a memorization leakage issue. In this case study, I will apply the Normative Ethical Theory of Utilitarianism to the issue of training data extraction. The issue at hand is that learning machines such as ChatGPT rely on training data that has been scraped from publicly open sources, these sources usually contain information that is meant to be private and it is very difficult for developers to filter through and sort out the data before training. Software developers usually just hope for the best and hope that the learning machine (ChatGPT) isn’t triggered into releasing such private information. Training data extraction attacks have countered this idea of optimism and have become a real issue for models such as ChatGPT. This issue now challenges software developers and raises the question: whether they should value training learning machines to be powerful or should they value human privacy and limit the use of sensitive data? I argue that according to the theory of Utilitarianism, the use of sensitive data should be limited when it comes to training learning machines such as ChatGPT and that human privacy should be valued and respected. I will explain the key ideas of Utilitarianism, then I will further describe the issue and severity of data extraction attacks, and lastly I will explain how the theory of Utilitarianism can solve this issue.
Utilitarianism is a theory that is a particular version of Consequentialism, this means that out of all the actions you could perform, the action you ought to perform is the one that brings the most utility. By definition of Utilitarians, the main concept of the Utilitarianism principle is simply just bringing about ‘utility’. Although Utilitarianism is also interpreted as pleasure instead of happiness, and pain instead of unhappiness it all focuses on the concept of utility. For Utilitarians, the word ‘utility’ is simply just a fancy word for happiness and pleasure and also the concept of Utilitarianism can be used as a mathematical way of thinking about morality because this way you can view the concept of utility as a number and that number is how much an individual is willing to pay or give up for a specific outcome or sometimes to avoid a specific outcome and avoid disutility. Utilitarians are able to decide whether an action is morally wrong or right by determining if the action is desirable and helpful for a greater means. For Utilitarians if an act is intrinsically desirable, then it is good and whatever is intrinsically undesirable is evil.
According to the Github article, “Training data extraction is the phenomenon that if you train a machine-learning model (like ChatGPT) on a training dataset, some of the time the model will remember random aspects of its training data — and, further, it’s possible to extract those training examples with an attack (and also sometimes they’re just generated without anyone adversarially trying to extract them)” (Nasr et al). Simply put, machine learning models are developed and trained by using sensitive data from actual human information (the more information used and stored, the more powerful the machine) and in turn these machines store this data and can release said data at any time if the function is triggered such as telling the machine to repeat a word. The issue at hand with training data extraction attacks are like pieces in a puzzle in my opinion, all the outcomes trace back the first step. The first step is using sensitive human data to develop learning machines, the second step is difficulty and poor filtering of data, which leads to the third step of these models just being an open vulnerability waiting to be triggered into releasing the sensitive data.
According to the principles of Utilitarianism, developers of machine learning models such as ChatGPT should act in a way that is intrinsically desirable and value human privacy. Upon many research articles proving the vulnerability of ChatGPT, this emphasizes the need for developers to step in to limit the use of sensitive data. Examining this issue from the mathematical scale of Utilitarianism, the solution in this issue relies on how much stored data developers are willing to sacrifice in order save the world from data breaches, and constant information leaks. According to Utilitarianism, Developers must weigh the good and the bad and come to a decision that brings about the most utility. On one hand, learning machines like ChatGPT provide benefits such as boosting productivity and efficiency in any and most tasks, while on the other hand, the vulnerability these models hold can cause harm by allowing events such as data breaches, identity theft, and random information leakage all to be more frequent.
To resolve this issue, developers should limit the use of sensitive human data to make these learning machine models more powerful because this would bring about the most utility for all of mankind. Upon all the research and weighing the benefits of machine learning to the severity of harm humans would be in, developers ought to make the decision that brings about the most utility for mankind which is to value human privacy over creating powerful machines. In this case the utility lies in the prioritization of human privacy because by the theory of Utilitarianism, one can come to the conclusion that the harm that may be endured from data extraction attacks is far greater and also outweighs the benefits of having powerful learning machine models like ChatGPT. Using Utilitarianism to solve this issue calls for developers to limit the use of sensitive human data when developing these learning machine models.
In sum, the issue of training data extraction attacks machine learning models experience can be resolved by following the principles of Utilitarianism. My argument that the use of sensitive data should be limited when it comes to training learning machines such as ChatGPT and that human privacy should be valued and respected is supported by the principles of Utilitarianism. The act of valuing mankind is what brings about the most utility in this case study and disutility would mean developers sit back and allow for events such as data breaches, identity theft, and random information leakage to keep happening. I like the theory of Utilitarianism in this case study because it should remind developers that they are also human and it is simply fucked to sit back and not do anything knowing you are putting other humans at risk of harm.