Cybercriminals won’t quit.
Neither will Luddy School of Informatics, Computing, and Engineering researchers.
In the middle of this latest good actor vs. bad high-tech battle comes more difference-making Luddy research. Featured in a paper co-authored by XiaoFeng Wang associate dean for research and James R. Rudy Professor of Computer Science, and Xiaojing Liao, assistant professor of Computer Science, as well as Ph.D. students Jian Cui and Zilong Lin, it brings attention to the vulnerabilities surrounding Large Language Models and offers possible solutions.
The paper, “Malla: Demystifying Real-world Large Language Model Integrated Malicious Services,” reflects a first-of-its-kind study showing cybercriminals are already misusing and manipulating LLMs.
Large Language Models such as OpenAI ChatGPT, Pygmalion-13B and Claude-instant have huge benefits and potential in processing and generating language-based content, but have weaknesses that cybercriminals exploit to generate bogus websites and phishing scams, write malware code, and produce false and misleading images.
OpenAI is targeted the most.
This is nothing new – cutting-edge technology is created that has amazing benefits, cybercriminals devise ways to exploit it, and researchers work to stop them.
“This has happened in the past,” Wang said, “and will happen in the future.”
How bad is the current LLM problem, which Liao called a “critical area of concern?” There was no way to tell -- until now.
Liao said their research shows cybercriminals “are using LLMs as malicious services to launch sophisticated spam email campaigns and other cyberattacks (such as) ransomware.”
Their research analyzed and characterized the “real-world behavior” of such criminals through services such as BadGPT, XXXGPT and Evil-GPT. They collected 13,353 listings from nine underground marketplaces and forums such as BlackHatWorld, Hack Forums and BreachForums from Nov. 30, 2022, through Oct. 12, 2023.
DarkGPT and EscapeGPT were among those capable of producing “high-quality malware” that evaded detection and security measures.
Their research offered guidelines to responsibly develop and deploy LLM technologies.
“By identifying these risks and developing effective mitigation strategies,” Liao said, “we aim to ensure that LLMs are utilized in ethical and beneficial ways.”
Wang said LLMs escalate the cat-and-mouse battle between cybercriminals and those who want to defeat them. LLMs provide a low-cost way for cybercriminals to build evasive malware, realistic spam and misinformation. Their study publicized the risk, and explored how cybercriminals exploit LLMs and how researchers can develop defenses.
The good news -- LLMs need a lot of resources to train. Researchers have those resources. Cybercriminals typically do not.
“In this sense,” Wang said, “the defender could be in a better position in the arms race.”
He added that the problem comes in researchers’ current limited understanding of LLMs, which makes it “harder to prevent all possible abuse of the technique.”
Their research has generated plenty of buzz, including a recent story in Techpolicy.press, a nonprofit media and community venture designed to provoke new ideas, debate and discussion at the intersection of technology and society.
Liao said she’s cautiously optimistic about reducing these threats.
“The rapid advancements in this field are improving our ability to design and regulate these systems more effectively,” she said. “Collaborative efforts between academia, industry, and policymakers are crucial in this endeavor.”
Their research improves the likelihood prevention is coming. Liao called the contributions of Cui and Lin “pivotal.”
“I see a lot of opportunities for technological innovations,” Wang said, “which could help build not only more effective, but also more secure AI in the future.”