It got worse in terms of the security of the code being created, as those in the AI group were "significantly more likely to provide an insecure solution" or use trivial ciphers to encrypt and decrypt strings. View all newsletters Sign up to our newsletters Data, insights and analysis delivered to you By The Tech Monitor team Sign up here AI coding assistants: use with caution It involved a relatively small number of participants with varying levels of experience including undergraduate students and industry professionals using a purpose-built app that was monitored by the administrators. Only three programming languages were used in the project Python, C and Verilog. “Additionally, we found that participants who invested more in the creation of their queries to the AI assistant, such as providing helper functions or adjusting the parameters, were more likely to eventually provide secure solutions.” The team wrote in their paper: “We observed that participants who had access to the AI assistant were more likely to introduce security vulnerabilities for the majority of programming tasks, yet also more likely to rate their insecure answers as secure compared to those in our control group.”
They said that those using assistants are "delusional" about the quality of that code.
The researchers from Stanford wanted to find out whether users "write more insecure code with AI assistants" and found this to be the case. They all had access to a web browser to search for help but only 33 had the AI assistant.ĪI assistant tools for coding and other tasks are becoming more popular, with Microsoft-owned GitHub launching Copilot as a technical preview in 2021 as a way to “improve developer productivity”. They had to perform five security-related programming tasks including ones to encrypt or decrypt a string using a symmetric key. The study involved a group of 47 developers, 33 of whom had access to an AI assistant while writing code, while 14 were in a control group flying solo. GitHub introduced its Copilot AI assistant in 2021 and it is widely used by developers to “improve productivity” (Picture courtesy of Postmodern Studio/Shutterstock) One AI expert told Tech Monitor it’s important to manage expectations when using AI assistants for such a task. Artificial intelligence-based coding assistants like GitHub’s Copilot leave developers “deluded” about the quality of their work, resulting in more insecure and buggy software, a new study from Stanford University has found.
0 kommentar(er)
