When considering the many things that AI may be able to do for us all, there’s both a lot of promise and a lot of vulnerabilities that we must consider. Perhaps one of the most concerning vulnerabilities that AI creators must be concerned with is insecure plugin design in Large Language Models (LLM). Here, let’s talk about some of the key identity challenges facing AI-powered applications.
Data Security Challenges
When concerning oneself with the security of any system, one of the largest concerns that we can have is data security. The reason for this is simple: we’re used to routinely storing vast amounts of data on drives and cloud storage, and a lot of this data is often sensitive.
In the case of AI-powered applications, giving the AI app access to this much data is generally considered to be a good thing. It can allow the app to interact with an increased number of applications and users if it is more, for want of a better term, knowledgeable.
However, all the data to which the AI has access must travel back and forth from the data drive to the AI itself. This data may also take journeys to separate databases for storage or verification or to plugins for additional actions to be carried out. At every link in this chain of data transfer, there is a potential data security challenge. A robust system will be secure at every step along the way, but it can be difficult to create a system that is secure in this way. This is a challenge for the use of sensitive data in and with AI-powered applications.
Excessive Agency
Another challenge that AI-powered applications can face in the world of identity and authorization is the issue of excessive agency. This can particularly be an issue in the case of adaptive authentication, where AI is tasked with risk-evaluating an access request in real-time.
In that situation, AI may be given additional agency over the access request and any credentials that the user has provided in order to expedite the process or avoid an access request getting stuck in a loop. At this time, the AI model can have some level of decision-making autonomy, which it may use to access a number of different plugins for different uses.
If the AI is given an excessive degree of agency in order to allow or disallow access, then it may share sensitive data with an unsuitable function or plugin. At the end of the day, an AI model doesn’t inherently understand the sensitivity of certain documents – it must be taught.
A great way to avoid this being an issue would be to use a whitelist system to give the AI agency at different levels. By default, it has the agency to do nothing. Then, with careful incrementation and development, you can whitelist certain abilities and plugins.
Privilege Escalation
Generally speaking, privilege escalation isn’t an issue for the user but rather an issue for the company or individual running the AI. This security breach takes the form of the user manipulating the AI in order to get a higher level of privilege access to data that the AI can access. In theory this could be done in a number of ways, but would likely be carried out through prompt injection techniques to fool the AI assistant itself.
After escalating their privilege to the level of a system administrator or a high-level member of the company concerned, the bad actor then has access to a wealth of sensitive data. This could include proprietary data that a company holds copyright over, or it could also include sensitive information on employees, such as addresses and bank details.
A good way to avoid this being an issue is to practice good data sanitization. By this, we mean that you should avoid allowing your AI’s data set to include anything that you wouldn’t want to get into the public eye. This may seem like overkill, but in the case of employee safety and proprietary data maintenance, overkill is often the best course of action.
There are a number of challenges facing the world of AI-powered applications, but with a little skill and forethought, it’s possible to overcome them.
