OP 24 February, 2024 - 02:59 AM
A new study from researchers at the University of Illinois at Urbana-Champaign (UIUC) has shown that large language models (LLMs) can be used to hack websites without human intervention.
The study demonstrates that LLM agents, using tools for API access, automated web surfing, and feedback-based planning, are able to independently detect and exploit vulnerabilities in web applications.
As part of the experiment, 10 different LLMs were used, including GPT-4, GPT-3.5 LLaMA-2, as well as a number of other open models. Testing was conducted in a sandboxed environment to prevent any real damage, on target websites that were tested for 15 different vulnerabilities, including SQL injection, Cross Site Scripting (XSS), and Cross Site Request Forgery (CSRF). ). The researchers also found that GPT-4 from OpenAI showed successful completion of the task in 73.3% of cases, which significantly exceeds the results of other models.
One explanation given in the paper is that GPT-4 was better able to change its actions depending on the response received from the target website than open source models
The study also included an analysis of the cost of using LLM agents to attack websites and comparing it with the cost of hiring a penetration tester. With an overall success rate of 42.7%, the average cost of a hack would be $9.81 per website, which is significantly cheaper than a human expert ($80 per attempt).
The authors of the paper also expressed concern about the future use of LLMs as autonomous hacking agents. According to the scientists, while existing vulnerabilities can be detected using automated scanners, LLM's ability to be hacked independently and at scale poses a new level of danger.
Experts called for the development of security measures and policies that promote the safe exploration of LLM capabilities, as well as the creation of an environment that allows security researchers to continue their work without fear of being penalized for identifying potentially dangerous uses of models.
OpenAI representatives told The Register that they take the security of their products seriously and intend to strengthen security measures to prevent such abuse. The company's specialists also expressed gratitude to the researchers for publishing the results of their work, emphasizing the importance of cooperation in ensuring the safety and reliability of artificial intelligence technologies.
do you think they will replace penetration testers with AI's soon ?
The study demonstrates that LLM agents, using tools for API access, automated web surfing, and feedback-based planning, are able to independently detect and exploit vulnerabilities in web applications.
As part of the experiment, 10 different LLMs were used, including GPT-4, GPT-3.5 LLaMA-2, as well as a number of other open models. Testing was conducted in a sandboxed environment to prevent any real damage, on target websites that were tested for 15 different vulnerabilities, including SQL injection, Cross Site Scripting (XSS), and Cross Site Request Forgery (CSRF). ). The researchers also found that GPT-4 from OpenAI showed successful completion of the task in 73.3% of cases, which significantly exceeds the results of other models.
One explanation given in the paper is that GPT-4 was better able to change its actions depending on the response received from the target website than open source models
The study also included an analysis of the cost of using LLM agents to attack websites and comparing it with the cost of hiring a penetration tester. With an overall success rate of 42.7%, the average cost of a hack would be $9.81 per website, which is significantly cheaper than a human expert ($80 per attempt).
The authors of the paper also expressed concern about the future use of LLMs as autonomous hacking agents. According to the scientists, while existing vulnerabilities can be detected using automated scanners, LLM's ability to be hacked independently and at scale poses a new level of danger.
Experts called for the development of security measures and policies that promote the safe exploration of LLM capabilities, as well as the creation of an environment that allows security researchers to continue their work without fear of being penalized for identifying potentially dangerous uses of models.
OpenAI representatives told The Register that they take the security of their products seriously and intend to strengthen security measures to prevent such abuse. The company's specialists also expressed gratitude to the researchers for publishing the results of their work, emphasizing the importance of cooperation in ensuring the safety and reliability of artificial intelligence technologies.
do you think they will replace penetration testers with AI's soon ?
![[Image: Fax9oq1.gif]](https://i.imgur.com/Fax9oq1.gif)
![[Image: heart5.png]](https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fstatic.cracked.sh%2Fimages%2Fsmilies%2Fheart5.png)
