OWASP Top 10 LLMs Vulnerabilities For Businesses In 2024
Harsh Sahu
CTO
February 2024 | 10 mins
The frenzy of interest in Large Language Models (LLMs) following the release of mass- market pre-trained chatbots in late 2022 has been remarkable. Businesses eager to harness the potential of LLMs are rapidly integrating them into their operations and client- facing offerings.
Yet, the breakneck speed at which development teams are adopting LLMs has outpaced the establishment of comprehensive security protocols, leaving many applications vulnerable to high-risk issues.
OWASP (Open Web Application Security Project) recently released the top 10 most critical vulnerabilities for businesses in LLMs. Each vulnerability is unique and requires a specific data security protection feature to provide a comprehensive solution.
Founded in 2001, the Open Web Application Security Project (OWASP) is a global, non-profit community dedicated to empowering developers and security professionals with free, open-source resources to secure web applications against cyber threats.
OWASP top 10 is a regularly published report which highlights online security concerns on web application security, focusing on the top 10 vulnerable tasks.
The OWASP LLM top 10 vulnerabilities for businesses are explained below:
Prompt Injection Vulnerability occurs when an attacker manipulates a large language model (LLM) through crafted inputs, causing the LLM to unknowingly execute the attacker's intentions.
This can be done directly by "jailbreaking" the system prompt or indirectly through manipulated external inputs, potentially leading to data exfiltration, social engineering, and other issues.
Let’s understand direct and indirect prompt injections:
Additionally, indirect prompt injections do not need to be human-visible/readable, as long as the text is parsed by the LLM.
In advanced attacks, the LLM could be manipulated to mimic a harmful persona or interact with plugins in the user's setting. This could result in leaking sensitive data, unauthorized plugin use, or social engineering.
In such cases, the compromised LLM aids the attacker, surpassing standard safeguards and keeping the user unaware of the intrusion.
Insecure Output Handling refers specifically to insufficient validation, sanitization, and handling of the outputs generated by large language models before they are passed downstream to other components and systems.
Successful exploitation of an Insecure Output Handling vulnerability can result in XSS and CSRF in web browsers as well as SSRF, privilege escalation, or remote code execution on backend systems.
Training data poisoning refers to manipulation of pre-training data or data involved within the fine-tuning or embedding processes to introduce vulnerabilities (which all have unique and sometimes shared attack vectors), backdoors or biases that could compromise the model’s security, effectiveness or ethical behavior.
Poisoned information may be surfaced to users or create other risks like performance degradation, downstream software exploitation and reputational damage. Even if users distrust the problematic AI output, the risks remain, including impaired model capabilities and potential harm to brand reputation.
An attacker interacts with an LLM in a method that consumes an exceptionally high amount of resources, which results in a decline in the quality of service for them and other users, as well as potentially incurring high resource costs.
Furthermore, an emerging major security concern is the possibility of an attacker interfering with or manipulating the context window of an LLM.
This issue is becoming more critical due to the increasing use of LLMs in various applications, their intensive resource utilization, the unpredictability of user input, and a general unawareness among developers regarding this vulnerability.
The supply chain in LLMs can be vulnerable, impacting the integrity of training data, ML models, and deployment platforms. These vulnerabilities can lead to biased outcomes, security breaches, or even complete system failures.
Traditionally, vulnerabilities are focused on software components, but Machine Learning extends this with the pre-trained models and training data supplied by third parties susceptible to tampering and poisoning attacks.
LLM applications have the potential to reveal sensitive information, proprietary algorithms, or other confidential details through their output.
This can result in unauthorized access to sensitive data, intellectual property, privacy violations, and other security breaches.
It is important for consumers of LLM applications to be aware of how to safely interact with LLMs and identify the risks associated with unintentionally inputting sensitive data that may be subsequently returned by the LLM in output elsewhere.
LLM plugins are extensions that, when enabled, are called automatically by the model during user interactions.
The model integration platform drives them, and the application may have no control over the execution, especially when the model is hosted by another party. Furthermore, plugins are likely to implement free-text inputs from the model with no validation or type-checking to deal with context-size limitations.
This allows a potential attacker to construct a malicious request to the plugin, which could result in a wide range of undesired behaviors, up to and including remote code execution.
The harm of malicious inputs often depends on insufficient access controls and the failure to track authorization across plugins. Inadequate access control allows a plugin to blindly trust other plugins and assume that the end user provided the inputs.
Such inadequate access control can enable malicious inputs to have harmful consequences ranging from data exfiltration, remote code execution, and privilege escalation.
An LLM-based system is often granted a degree of agency by its developer - the ability to interface with other systems and undertake actions in response to a prompt.
The decision over which functions to invoke may also be delegated to an LLM 'agent' to dynamically determine based on input prompt or LLM output.
Excessive Agency is the vulnerability that enables damaging actions to be performed in response to unexpected/ambiguous outputs from an LLM (regardless of what is causing the LLM to malfunction; be it hallucination/confabulation, direct/indirect prompt injection, malicious plugin, poorly-engineered benign prompts, or just a poorly-performing model).
The root cause of Excessive Agency is typically one or more of: excessive functionality, excessive permissions or excessive autonomy. This differs from Insecure Output Handling which is concerned with insufficient scrutiny of LLM outputs.
Overreliance can occur when an LLM produces erroneous information and provides it in an authoritative manner.
While LLMs can produce creative and informative content, they can also generate content that is factually incorrect, inappropriate or unsafe. This is referred to as hallucination or confabulation.
When people or systems trust this information without oversight or confirmation it can result in a security breach, misinformation, miscommunication, legal issues, and reputational damage.
This entry refers to the unauthorized access and exfiltration of LLM models by malicious actors or APTs.
This arises when the proprietary LLM models (being valuable intellectual property), are compromised, physically stolen, copied or weights and parameters are extracted to create a functional equivalent.
The impact of LLM model theft can include economic and brand reputation loss, erosion of competitive advantage, unauthorized usage of the model or unauthorized access to sensitive information contained within the model.
The theft of LLMs represents a significant security concern as language models become increasingly powerful and prevalent.
Organizations and researchers must prioritize robust security measures to protect their LLM models, ensuring the confidentiality and integrity of their intellectual property.
Employing a comprehensive security framework that includes access controls, encryption, and continuous monitoring is crucial in mitigating the risks associated with LLM model theft and safeguarding the interests of both individuals and organizations relying on LLM.
Few recommended strategies from OWASP in a data security tool for LLM risk mitigation:
Organizations are drowning in sensitive data: it's everywhere, hidden in Public Clouds and Big Data Solutions.
This exposes them to constant compliance risks and hefty fines from federal regulatory bodies like GDPR, CCPA, DPDPA, and local bodies like Virginia Consumer Data Protection Act, New York Privacy Act.
Transparency in data access and usage is the need of the hour. Organizations have to be stringent on who gets access to sensitive information. The duration and amount of data accessed is as critical as the amount of data seen by the user.
Attribute- based access control, dynamic data masking, and cleaning of data before feeding into LLMs reduces the chances of data breach to a large extent. This is where OptIQ comes in as the much needed tool for modern-age data security, with real-time alerts and notifications to elevate your data safety and protection.
OWASP top 10 can be mitigated through OptIQ both for users and LLMs. Sign up for a demo today !
The OWASP top 10 vulnerabilities for llm applications:
Few privacy concerns against LLMs are:
LLM are vulnerable for sensitive data as LLMs are considered to be black holes with no delete button. It is better to secure sensitive data in your training data by de-risking, cleaning and processing the data before feeding it to LLMs. Schedule a demo to understand how OptIQ can prevent sensitive data sprawl in your training data.
