How vulnerable are AI systems? The question may lead to disappointment to all those who have been excited using the tools so far. But yes, it is true. Lately, researchers at Anthropic Lab have uncovered vulnerabilities in the safety features of some AI platforms and could be potentially exploited for cybercrime or terrorism.
The findings reveal a technique called “many-shot jailbreaking.” It can be manipulated easily by flooding them with examples of harmful requests. The system can bombarded with a plethora of examples of illicit activities similar to building bombs or manufacturing drugs. The AI may eventually provide such instructions itself and this may bypass safety protocols.
Anthropic Lab is known for producing the large language model (LLM) behind Claude, which is believed to be a close competitor to ChatGPT. It emphasized that the attack method can force the AI systems to generate potentially harmful responses.
However, Anthropic claims that simpler AI models may not be susceptible to the exploit due to limited context window. The greater risk is with newer and more complex systems with larger context windows. It is suggested that these advanced models may also be quicker to circumvent their own safety rules.
Anthropic proposed some solutions to overcome the issue such as implementing mandatory warnings after user inputs to remind the system of its safety obligations. It is believed that at least the approach may affect the performance of the system in other tasks.
Anthropic Lab has shared the recent research with peers. It aims to address the vulnerability promptly to safeguard against potential misuse of AI technology.
